I'm not sure it's actually possible. WotC need to sell books, and they hit on "system mastery" as the way to do it - put out lots of options that interact in a myriad of ways, and people will buy into them in numbers in order to make the best characters possible.
Not really. The 3e approach is as you describe. The 4e approach isn't broken stuff. It's to expand the range of detailed playable options with every new splatbook that comes out. Brawler fighters aren't broken in any way - but they are really worth having and it's inspiring to see sword and fist done properly. You might have bought 3e splat books for broken stuff. You buy 4e ones for good balanced options (which might explain why fewer books were produced).
The problem is that merely putting out large numbers of options guarantees that you'll screw up somewhere and end up with broken options. The 4e solution to that was the ongoing stream of errata,
Honestly, most of that was that the PHB1 and DMG1 were seriously under-playtested.
I'm sure it would be possible to build a version of 3e (admittedly, heavily revised) in which the major discrepancies were eliminated. But the problem is that if I can slap together a character at random using just my PHB, and have it be close to as good as the best possible option, where's my motivation to spend another $1,500 on all the rest of the books to get that best possible option?
Because you can't slap a brawler fighter together with either the 3e or the 4e PHB and not have it be a bad joke. Neither edition's PHB will do Bravura (reckless) Warlords. Or self-harming invokers.
Sadly, there is no such edition. If the choices are "play on with a broken edition" or "don't play on at all" - given that there is no prospect of the edition ever being fixed - what do you choose?
I'd say that both oD&D and 4e are a whole lot less broken than 3e.
So I'm in the position where I have to reluctantly live with the problems. Despite its flaws, 3e remains the "best fit" system for me.
That's part of why the highlighting of flaws by people like the CharOp group (and also Edition Warriors on the other side) is so problematic - it's not that I can deny that those flaws exist, it's that they do exist and there's nothing I can do about them.
I'd say that's a
lot less frustrating than the 4e situation where the "flaws" highlighted by edition warriors are normally things that are straight up misconceptions about the game - and few of the anti-4e lobby seem willing to be corrected when they have flat out said what happens is the opposite of what the rulebooks say.
I never understand this viewpoint. A well-intentioned amateur should be just as good at a game as a seasoned expert?
No. It's degrees and degrees. A well intentioned amateur should be able to sit down at the table with a seasoned expert and neither look nor feel like a supernumerary. D&D is largely cooperative and the difference shouldn't be at the level where the amateur's PC makes very little difference as to whether the amateur is there or not. I'm always going to be a strong player - I don't want to make others feel pointless because of it.
Where the game is actually broken is when it takes options that should be equally valid, and makes some of them better than others. For example, a fighter gains more benefit from Intelligence (skills) or Wisdom (saves) than Charisma, which defies the notion of what most of us think as a fighter; they should be able to be great leaders and confident heroes as well as seasoned tacticians, but the rules don't support it. So you see a bunch of uncharismatic fighters, because taking rational steps to optimize your character leads you down that path. Character optimization leads to the same (boring) results, where the game should be encouraging diversity and nuanced choices. That's the game being broken.
Once more I say you mean
3e character optimisation leads to the same (boring) results. With 4e the range is narrow enough and there are enough benefits given with most concepts that two 4e fighters can look almost as different as a 3e illusionist who's banned evocation and conjuration, and a 3e evoker who only casts direct damage spells. Probably more different given that both the wizards wear simmilar armour and cast from a spellbook while one of the fighters can be a rapier and dagger armed burglar with more skill proficiency than a 3e rogue and wearing studded leather armour - and the other a polearm wielder who sends the enemy flying while wearing scale. Given that the best way to optimise in 4e is to take any concept and turn it up to 11, it does not lead to the same (boring) results unless the concept is "highest dpr".
What is being asked for is that D&D, despite being a game should not do that.
Fallacy of the excluded middle. What is being asked for is that the edge
achieved in character creation should be
small. You'll always get a reward for system mastery if there are any choices to be made. Getting rid of it is impossible - and undesirable.
And in 4e the direct competative edge is fairly small. The reward for system mastery is the ability to easily make offbeat concepts that are competative. To take one trivial example, I believe Frank Trollman has been known to claim that you can't play a lightly armed spear and shield figher in the 4e PHB. This is entirely false - the options are there and very effective in the PHB, but you need system mastery to notice that powers such as Rain of Blows and Armour Piercing Thrust synergise with the spear and with high dexterity in such a way as to make up for the poor weapon and going for light armour. This is where 4e rewards system mastery. Beginners get decent characters using the options that are presented obviously (for a fighter that would be heavily armoured sword and board and two handed sword in the PHB), experts can either make their sword and board fighters about 10% stronger - or make less obvious builds such as polearm fighters or light armour, spear, and shield sing.
We haven't got rid of system mastery or even tried to. We've just made it so not having it doesn't reduce peoples' fun at the table.
Which makes the entire remainder of your criticism irrelevant.