Here's the crux: preferring a toolbox/generic style that lets you make every special snowflake you can imagine is all well and good, but there are tradeoffs to be made for doing that.
One big tradeoff from a design perspective is that this is newbie poison. If you want to give your game a broad, open appeal, you do not want to have to educate new players on the relative benefits of a pool of dozens upon dozens of options. People will just not bother - there are better things to do with their time than learning how to build an imaginary dragon elf or whatever and getting it to do what they want it to do.
Two points:
First, while I agree that
too many options can turn off newbies,
too few is also a problem because it makes it harder for newbies to take the image in their head and find the right mechanics to express it. Skilled players familiar with the system can find good mechanical approximations for most concepts, particularly by being aware of refluffing possibilities, but an inexperienced player only has the options in front of them at the moment. For example, a new, inexperienced player displeased with their spell options as a sorcerer may not even know that wizards get access to all the same spells, plus more. Even if they did know, they might reject switching to Wizard on fluff (or primary stat) grounds, not thinking to ask the DM to refluff the wizard as a sorcerer.
The sorcerer/wizard divide is particularly hard for new players because you're almost always picking your class by comparing a rough-hewn concept to the class descriptions, long before you've read through the spell lists. But those descriptions don't tell you which concepts the sorcerer class has the spells to mechanically support and which concepts are much better represented as wizards. (Unlike say, Cleric vs Druid where the thematic difference between their spell lists is readily apparent.) So in this case the relative lack of options on the sorcerer spell list is
more of a problem for newbies, simply because they don't that the limitations they're facing when expressing their sorcerer concept may be surmountable.
Second, increasing the size of the Sorcerer spell list to make more concepts realizable can't qualify as
too many options until it surpasses the wizard list, unless you're arguing that 5e wizards themselves are newbie poison.
From my perspective as a big ol' nerd, a big tradeoff is that generic classes with lots of options have an emphasis on building a character. Wading through a sea of options to curate the perfect mix just reeks of Paradox of Choice - none of these options are increasing my enjoyment of playing the character, where I won't be picking character options at all. In fact, given that some options will be better than others (because any group of two or more things can be ranked), there's a real risk of this turning into "not really a choice" or "a choice between being flavorful and being effective," and leaving me less satisfied than I would be if I had no choice!
Personally I feel the character-building minigame is an important part of what drives experienced players to keep playing D&D, because it can be done solo. When in-between campaigns and/or gaming groups, the ability to build characters as a creative exercise keeps interest in the game high. Once of the prices of having deliberately limited options is reducing the appeal/longevity of the minigame to your veteran fanbase.
None of that means your preference is wrong, but it does mean that not being super-duper flexible isn't some flaw with the class's design, but is an intentional decision that, in getting rid of the ultimate in generic flexibility, stands to significantly improve the class in the estimation of a broad swath of the gaming audience. To put that away just so someone can imagine a "light bender" and a "dimensional witch" and an "arcane thief" and a dozen other preciously unique options all with one class that must also be named "Sorcerer" and who is barred from offering these as archetype options....that's a pretty big ask a pretty marginal gain. That same result can be pretty easily realized in practice by throwing more subclasses at the existing sorcerer 'till yer face turns blue.
I don't understand how, for example, not giving the sorcerer access to the entire wizard list significantly improves the class for you personally. You'd have no more options than a wizard would, so there shouldn't be too many, and it leaves the fluff divide between the classes entirely intact, so they remain distinct. (If both classes had unique spells, I could see an argument that merging the spell lists would reduce the distinction between the classes, at least so long as the assignment of particular spells had a thematic divide. But when one class's spell list is a strict subset of the other's and lacks any rhyme or reason, I don't find that convincing.)
I'm also unconvinced that the "marginal gain" of improving the game for whatever-size swath of the gaming audience feels as I do is necessarily any less valuable than improving it for whatever-size swath shares your opinions. Without some sort of concrete data on the prevalence of certain preferences, what basis can there be for claiming your opinion is more important than mine? It does indeed sound like you're saying that my preference is wrong on the (unknowable) basis that it is less popular than yours.
You can ignore any fluff as long as the DM lets you.
...
Also, if you want to ignore fluff, you could be approaching this from the other direction. You can ignore fluff and make a wizard who doesn't use a spellbook and who is tough and strong, or an Arcane Trickster with a mysteriously magical origin, or whatever floats your boat. That's as easy as ignoring warlock pacts.
There is a difference between tossing out the "sworn & beholden" fluff for the warlock and changing the mechanics of the wizard class to no longer have a spellbook or be based on charisma. I've never yet encountered a DM who wasn't fine with rewriting fluff, but I've met several who were very hesitant to make even small mechanical changes. And for those who do organized play, the former is legal, but the latter is not.
I'm in a game with two sorcerers, and we've got more than enough options between us to avoid doubling up on mechanics with room to spare. My actual play experience says there's enough variety there. White-room speculation on all the hypothetical character types you miss out on doesn't affect the actual enjoyment of the class one little bit.
I'm glad! Your experience shows that the class has enough options for you to be content. But I don't see how your experience is sufficient to reduce my perspective to "white-room speculation". I am indeed frustrated with the sorcerer builds that the truncated spell list precludes--there's nothing hypothetical about it.