Presumably if 3e is pretty well balanced then AD&D, whose Mages and Clerics were considerably weaker than 3e made, is a game you consider badly balanced.
It's not really a relevant comparison. The relative power of one character class versus another isn't the main balance consideration in my eyes, as I explained in detail elsewhere. And even if it was, the differences between those two editions aren't all that large.
The equation of hit points with luck is found in Gygax's PHB and DMG.
So are many other things that have been revised, changed, or forgotten since then. Given that this thread is forward-looking and about what mechanics should be rather than have been (such as mechanics that subvert the hp as luck angle), I don't find the historical context particularly compelling.
I don't, which was my point. I know that Gygax's writings describe hit points that way. Gygax made a mistake. One which has hindered the progress of the game for the duration of its existence. No matter how you look at it, he made a lot of mistakes, not because he was incompetent, but because he was breaking new ground. He wasn't the Grand Poobah of roleplaying, simply the guy who wrote the first popular rpg. Many such mistakes (or evolutionary dead ends, if you prefer) have been fixed since then, and the game has expanded and evolved into new territory that he never could have anticipated. Why leaving hit points (in the mishmash of toughness and luck and skill conception) behind is such a big deal when so many other things have been changed I don't understand.
(4e develops this idea the furthest of any edition)
It regresses to an earlier stage of development in this and other ways by developing an idea that really doesn't warrant development. It would be like bringing back and further developing the weapon vs armor charts, or percentage-based thief skills. Or it would be like modern medicine abandoning drugs and surgery and perfecting the best possible bloodletting and exorcism techniques. Not a good idea.
Many RPGs have balanced mechanical effectiveness across PCs despite having no "inherent goals."
Balanced mechanical effectiveness at what?
But I am guessing that they approach the GM side of things - and particularly encounter building - in ways that are not your preferred approach.
They probably approach balance (which is generally on the GM side of things) in ways that are not my preferred approach.
This is absolutely and completely 100% false. Take a look in a 2e PHB. It has roles. The roles in question are IIRC Fighting Man (Fighter, Ranger, Paladin), Cleric (Cleric, Druid), Mage (Wizard, Illusionist (which had been demoted to a specialist wizard), other specialist), Rogue (Thief, Bard). And mysteriously they fit the archetypal four person party of magic user, cleric, fighter, thief in exactly the same way 4e does.
It has roles if you use the literal common-language definition of the word. All classes are roles in that sense, and they can be categorized somewhat as well in the way that 2e did. But
4e roles, which are specifically based on a reductionistic view of combat actions, are not there. 2e wizards were mages, not controllers, and those terms are not synonymous. As you point out, the mage category could include all wizard specialties, some of which are not focused on direct combat, and others of which can focus on virtually any type of combat action. Similarly, 2e rogues were not strikers (though some could be), priests were not leaders, and fighters were not defenders. Those are the roles that 4e postulates.
Versions of D&D prior to 4e also had warlords, leaders, powers, minions, and rituals, by the common language definition of those words, but 4e created new concepts and assigned those words to them, thus when used in an appropriate technical context, they all refer to things that exist only in 4e.
This is in no sense true either. It is possible to find people who consider any given edition to be balanced - but if the 1e PHB is balanced then the 2e one isn't
This is not true at all. It is does not follow that if one game is balanced all other games must not be; there's no universal standard for balance. They're all balanced in different ways. The 1e and 2e books are not (as you later get at) necessarily balanced for the same goals, nor do they necessarily use the same approach to achieve them.
Also I know very few people who consider 3.X is anywhere approaching balanced.
I know no one anywhere who has ever played 4e or would consider it a viable rpg, let alone a balanced one. Perhaps it may be that neither of us knows a statistically meaningful portion of the rpg community.
Balance is always balance to a specific purpose. The first question is what purpose it will be used for.
Well that makes sense, at least.
Balance between player character options is not irrelevant, but it certainly isn't the only type of balance. I find the obsession that some people have with this particular goal (making sure that every character creation choice is equally powerful) quite bizarre.
