They had multiple classes in 3.x, it was confusing and there was a ton of overlap. After a while they were making variations that were just flat out better than what had come before because there are only so many variations on the theme. It was difficult to keep track of all the different options and the core concepts were often much the same, subclasses achieve much the same results of flexibility without the complexity, overhead and confusion. People likely rejected adding more classes when they floated the idea early on because they remembered what a mess it was.
It is, of course, just an option chosen because there will always be compromises and limitations to what any one game can do. If you want more options there's always 3PP.