These are based on the state of the game at the time - for example I wouldn't ding AD&D for not having a unified resolution mechanic.
D&D versions
(This is subjective, some may disagree or have a workaround.)
5e: Large number of encounters expected to balance long-rest recovery mechanics.
4e: Too many distinct choices for mid and high level characters slows down combat across the board.
3.x: Design decision of prerequisite chains requiring planning out characters many levels in advance.
AD&D 2ed: Racial level limits would hard stop existing character advancement for some existing characters, which was a deal breaker when it happened.
Other RPGs
Star Wars d6 (WEG): Force users were so much more powerful then non-Force users
Deadlands (original): Six-shooters, while iconic, were so much less powerful then rifles that they weren't useful.
Rolemaster: Too many table look ups slowed play.
Mechwarrior (1986): Mech combat ability and everything else ability came from the same pools, so some characters were fantastic in mechs and useless elsewhere, and vice-versa. So neither side of play was satisfying to all.
Hero System (1989): Just because you can do something doesn't mean you should. In this case it was taking a system designed for vast flexibility for superheros (Champions) and using that as a universal system including gear-based and "hero-to-zero" play which weren't it's strong points at the time.
Amber Diceless: Hard to add players after campaign start because of whole-party bidding process character creation.
Warhammer Fantasy: High Toughness broke the damage balance between characters.
Dresden Files (2010): Thamaturgy was too subjective and complicated. (Actually, taking a streamlined but flexible system like FATE and trying to mechanically model all of the options in the Dresden Files books may have been the more overarching design flaw. FATE and high mechanical simulation is not the right match.)
13th Age: Overloads the ritual magic system to make up for very limited out-of-combat spell selection.