Whether the fighter was too weak or too powerful though, there was always the issue of how his fighting prowess improved....This made a Robin Hood-style archery contest awkward. Robin Hood-style sword-play was also awkward, as was Ivanhoe-style jousting....I thought 3E had solved the problem with its Fighter class and bonus feat list. At a glance, I saw that you could build an archer, a knight, a samurai, etc. It didn't really play out that way though.
It didn't I think for one primary reason - the designers had come up with a simple elegant system and it was so simple and elegant that they loved it as a formal designist solution and couldn't bring themselves to change it. The reasoning seemed to be, "But this is so beautiful and elegant, it has to be right." But off on the other side, the Wizard is hardly beautiful and elegant - it's the same crazy complex design that organicly grew to allow that literary range of apprentice to deus ex machina plot device we see in wizards. They ported the Wizard in from earlier editions out of the general philosophy of trying to attract back their former 1e loving fans.
They created...
1) A simple elegant solution.
2) A new innovative system.
3) An addition to a class already considered by many one of the strongest in the game.
All of that added up to being overly conservative. Compare what they did to the fighter, with what they did for the Cleric - previously considered a somewhat weak and boring class useful mainly as a hit point battery.
If you look at the 3e fighter, it's clear in three ways that it can't keep up with the Wizard in the long run. It's a remarkably good class up until about 6th level, but then Wizards start adding more and more spells, more and more powerful spells, and those spells that they have already known become more and more powerful as they increase in caster level. But the Fighter's simple elegant LINEAR design does none of these things. He doesn't get more feats as he levels up. The feats he does get access to don't significantly increase in power. And the feats that he's already taken don't scale up. Up until maybe 11th level, the inherent strengths in the class still roughly compensate, but when the 6th level spells start arriving, it's really over for the fighter except as a support character and henchmen for the parties spell casters.
Fundamentally there is nothing wrong with the idea of customizing the fighter to become awesome at something, and in fact it's actually pretty brilliant. It just didn't go far enough, especially when they were making the Wizard simplier and more elegant by removing most of 1e's balancing features on spellcasting. The problem was that you couldn't actually make the fighter awesome because you didn't have enough resources to spend.
I think that by the time 3e went to press, Monte already knew something was wrong with the existing system. And we see him toying with an additive solution in the form of a 'Prestige Classes'. By the time Sword and Fist is released, solving the problem of the fighter not having enough feats is firmly centered around the idea of the 'Prestige Class'. But for the most part, usable Prestige Classes - for any base class - turn out to look a lot like the base class with a feat every level. Only, unlike the fighter base class, that feat isn't generally customizable. You take a fixed progression of feats that are exclusive to your class. This just isn't a satisfying solution, and its really far from their initial elegant design. Worse, they create 'Prestige Classes' that are 'base class + feat every' level for the arcane classes as well, losing whatever balance they might have otherwise gained.
I believe the Prestige Class was far and away the worst design decision of 3e, and it really sowed the seeds for the systems eventual self-destruction. It was the wrong approach. I haven't fully solved the problem, but I knew I was at least partially achieving my goals when a player of RAW 3e played my game and had the revelation, "You don't take a prestige class... you BECOME a prestige class."
This is particularly odd when we look at the source material and see how many fantasy characters should be multi-classed human fighter/thieves, which the rules didn't (really) allow.
Someone on the boards made the observation that 1e AD&D had got the multiclassing system backwards. Humans should have been allowed to be multiclassed, and demihumans allowed to dual class. I think that's a particularly insightful observation if you are going to fix 1e while retaining its basic character. However, I believe that 3e is ultimately more elegant, and any fix of 1e would adopt so much of the philosophy of 3e that you might be better off porting what you wanted to keep of 1e into 3e than the reverse.
Also, the rules didn't lend themselves to such ad hoc calls; they didn't provide "hooks" for variations on the normal to-hit, then damage routine. If you want an archery contest, or a joust, or a light-saber battle, where do you begin?
AD&D begins as a tactical skirmish level wargame, and at that level it does it pretty good job of simulating the visuals of that sort of chaotic bloody melee from literature or cinema. But it has never done a good job of simulating the ebb and flow of a cinematic one-on-one duel or contest. It just didn't evolve out of that, and it never really considered it until too late.
I think you actually do not so bad with an archery contest, especially by the time you get to 3e and fighters of similar levels can so greatly distinguish themselves from each other in terms of skill with a bow. The problem you might have left is that in archery contests in the movies or stories, luck doesn't seem to play a role - it is foreordained that the hero cannot lose this contest regardless of the luck which his rival or the villain might have. The villain can 'roll a 20' (and as a trope, always does), and the hero will STILL win. And I think that this points to one of the tropes you might consider as part of your solution to balancing the mundane classes with the spellcasters - Mundane heroes always succeed when they have to. They seem to have a pool of resources that lets them 'roll a 20' when they have to, and they generally seem to have greater access to this pool than there supernaturally powered colleagues (who must rely on their powers). In other words, we need to at least consider a mechanic for simulating 'luck' or 'destiny' and we need to consider making this mechanic favor the mundane classes.
However, I agree with you fully that the cinematic 'duel' isn't really easy to do even in 3e - especially the duel that is to 'first blood'. You can work it, sorta, but really only if all heroes are built with the feats that let them be effective duelist. That being said, realistic first blood duels rarely last more than a few seconds anyway. And some source material, say John Carter, has gorier duels typically marked by a large number of small pricks and flesh wounds over the course of the battle. But we also want to capture the long panning shot of swords flashing that marks the classic bloodless fencing duel of cinema, we are going to need to do something. And we are going to have to do I think is as much about extending the combat system as it is about fixing a class. So again, I think the solution has to come from multiple directions - more options in combat, more restrictions on spellcasting, more resources for the mundane classes.
As far as providing more explicit options in combat, I think both 3e and Pathfinder are on the right track. But the problem typically is that they both try to extend the combat system via the feat system, providing feats that add options. The proper approach I think is to add the the option to the combat or skill system, and then provide a feat that makes you good at it. If you look at Ultimate Combat or the 3.X fighter splatbooks, so many of the feats should never have been feats at all. Instead, they should be interacting with existing combat and skill mechanics or simply added directly to the combat and skill options so that the everyone is expanded in options without the need for a feat and every class with good BAB and the right skills gains on the caster classes.
