Lots of things...
1) No usable skill system. The NWP system cludged into the game had far too many limitations.
2) Monster base attack bonus capped off too early, resulting in PC's easily able to face just about anything once thier AC got high enough.
3) Monster hit points were generally too low relative to the ammount of damage that a PC party could generate. This made rolling initiative at times seem to be the middle phase of combat, and the first round the end step.
4) AC's capped off at -10, resulting in high level fighters generally being able to hit everything with every attack.
5) Clerics and thieves were too weak - especially at high levels - relative to fighters and M-U's. There was absolutely nothing that a thief was particularly good at at high levels, and party clerics had no role other than healing more useful members of the party.
6) There was no good way to handle invisibility. PC's only faced a -10 penalty to hit invisible creatures, which was generally insufficient to keep high level PC's from hitting invisible targets with every swing.
7) There were too many combat subsystems (regular, unarmed, psionic) and far too many tables.
8) Humans were generally inferior to every other race, and the attempt to balance this by capping max levels of othe races was largely ineffective and annoying to boot.
9) Too many obvious things were left completely up to DM fiat. For example, there was no explicit Scent ability. In fact, I would say that if there was one single event responcible for me leaving 1st edition, it was an argument over whether or not Dire Wolves had a sense of smell with a rules lawyer. The PC claimed that - because the rules didn't explicitly grant the wolves the ability to smell things - that they couldn't. Not only was it the most ridiculous rules lawyering I'd ever been a part of, but it just pointed out how much the game was completely reliant on my good judgement at times when a good rules system ought to be giving explicit guidance.
10) Monsters and PC's used completely different rules systems, resulting in absurdities. For example, as the game began to mature and NPC's belonging to PC races began to be allowed to live by the same rules as PC's, it became increasingly illogical that races like say Hobgoblins which largely where still treated as 1+1 HD monsters had not been exterminated completely. On the other hand, if PC races were still treated as 1 HD monsters, then the gulf between PC power and NPC power frequently proved unworkable. How did normal people defend themselves from criminals, for example?
11) Monsters lacked ability scores, resulting in increasing strangeness as the game began to mature. For example, PC's (with DEX scores) generally had superior initiative scores over just about everything.
12) The official game had begun to resemble a vast collection of house rules. Virtually every DM was running a different game, and in some cases a radically different game. There was alot of good stuff out there, but not every DM was aware of it. Moreover there were important aspects to the original game (such as weapon vs. AC modifiers, shields effects on 'reflex' saves) that many DM's - even experienced ones - simply never understood, or knew about, or found too much trouble to use despite thier importance in keeping the game balanced.
13) Monsters AC was not explicitly broken out into which portion was due to armor and which portion due to other modifiers (like size, speed, dexterity, etc.). This increasingly made it difficult to rule how rules that interacted with AC effected monsters.
14) Players began to outstrip every monster in the books by about 12th level, and only by using monsters that broke the rules in various ways (usually with badly written special abilities, for example the 1st edition Korred's laughter ability) or making up your own creatures could you keep challenging players. High level play meant, '9th level and above'. The game's 'sweet spot' of between 2nd level and say 8th level was too small.
15) Too many of the original rules had been written from a highly repressive gamist standpoint where it was clear that Gygax himself was metagaming to 'keep the PC's in thier place' . The rules on magic item creation for example were so harsh that one could reasonably expect that no magic items would exist in a world ran strictly 'by the book'. Not enough characters of sufficiently high level would exist to create them, and what they would create would remain strictly for thier own use, and given the rules for item destruction would probably not last through thier lifetime much less much beyond it. There were not enough 13th level M-U's in existance to explain the abundance of swords +1, to say nothing of other magical equipment. The economic system and many other aspects of the game were written from the same 'DM vs. the PC's' perspective which always seemed to assume that if the DM wasn't continually arbitrarily beating the player's down that they'd go out of control. The game lacked non-narrativist means of keeping balance, so it opposed narrativist means as if they were game mechanics.
16) Low level M-U's simply didn't have much to do. A 1st level M-U was one of the most boring things you could possibly play. One spell. A horrible AC. Irrelevant attack capability. You basically stood back and hid through 90% of all encounters. Things didn't get much better until upper mid-levels, at which point you suddenly started to become the beatdown. But, by this point, you'd already missed much of the games sweet spot.
17) Horribly unbalanced classes. Paladin and Ranger were 'balanced' on the grounds that noone who didn't cheat when rolling up characters could possibly qualify. The number of 1st edition Paladin's generated by the 3d6 method of generating scores (or even the 4d6 method) without cheating can probably be counted on one hand - and that's being generous.
18) Most ability scores - say in the range of 8-14 - generated neither significant penalties nor significant bonuses. Characters with ability scores of 15+ (which were rare under the 3d6 method and not assured under 4d6) had major and exponential advantages over characters with say 6 13's or 14's.
By the time I quit 1st edition, I'd made extensive changes to the game.
a) I'd begun to give ability scores to every monster I used. This gave monsters the ability to compete with PC's in terms of THAC0, allowed monsters to use the NWP rules, allowed monsters to compete with NPC's in terms of initiative and in contested checks, and so forth.
b) I'd begun to break AC into 'Armor Class' and 'Armor Bonus' (a score that modified the opposing to hit role). Although your AC couldn't exceed -10, you could have a -6 AC and a +8 AB (which counter-intuitively gave opponents a -8 on thier 'to hit' rolls), with the net effect of increasing the cap on maximum AC. This let me introduce monsters which had effective AC's high enough that high level PC's would have a hard time hitting them. It also let me apply the 'weapon vs. AC' modifiers consistantly, so that weapons not intended to do damage to armored targets were not used as a such (and conversely Heavy Picks were wonderful against highly armored targets).
c) I'd begun using a primitive DR system, in that creatures with an AC higher (err lower) than 0, had what would now be classified as DR X/-. Again, this let me have durable monsters without radically increasing hit points.
d) I was giving thieves radically better NWP progression (inspired by NWP's in the 2nd edition Thieves Handbook).
e) I was using the famous 'critical hit' tables. I even had primitive 'keen' weapons that gave a +5% bonus on the chance to have critical hits. I'd taken swords of sharpnes and instead of simplying lopping off limbs (for which no mechanic existed for much of 1st edition), I'd made them do automatic criticals on every modified roll of 20+.
f) Much of the game only worked when you made alot of interpretation of the rules in ways that the designers simply never intended. For example, the XP awards table seemed to assume that a monster was eligible for only one award for an exceptional ability, but I found that they worked ALOT better if you granted the PC's an XP award for each exceptional ability that it had. Otherwise, an easy monster and a radically tougher monster would grant exactly the same XP.
And so forth. The system was still ridiculously obscure and unintuitive (a bonus to your AC lowered it for example), and it had never occured to me how I might clean it all up. What I did come to believe is that the system was just too primitive to bother cleaning up. I quit.
Then came 3rd edition, and I literally felt like I was reading someone that had taken my house rules and cleaned them up with more skill than I ever could have managed myself. Every page felt like either someone has taken my house rules and gone, "Hey, that's a good idea.", or it felt like I was reading the house rules of some DM who had had EXACTLY the same problems I had had as a DM and I was left thinking, "That's sooo coool. Why didn't I think of that? Why did it take us this long to think of that?"
I almost did it, but I like Eberron to much. I think if I do a d20 Modern game, I'll shoot for running a game with just that book and see how far I can go before getting another book.
