You don't create "diversity" by taking away or limiting players' abilities to make choices.
You most definitely can create diversity by limiting players abilities to make a choice. You may dislike that method but it accomplishes the goal.
You want players to play fighters who aren't focused on Strength? You need to figure out why they don't. The obvious answer is because the game is designed to mechanically support fighters with higher Strength scores.
What if I think strength and con are the most important stats for a fighter and that they should be more important for any fighter than intelligence or charisma
Period. If you limit their primary ability, you are effectively limiting the potential for that class they had chosen to play unless you provide the means for that character to fulfill its potential with a different ability score.
If your point is that such characters wont be as strong as their 20 stat cap brethren then that's true. I'm failing to see why characters being slightly weaker than what's currently out there now is an argument for not doing this?
Dexterity already does this because there are a number of options and features, like finesse and ranged attacks, to make it viable for that class (and others).
All the stats have a use. Some stats are more important to some classes, but as long as you start with a 14 to 16 you can make do till you develop a different plan for one of those other stats. Maybe it's multiclassing into bard or sorcerer or warlock. Maybe it's making a really hardy con based fighter.
But suppose you create similar options for other characteristics, like Intelligence? 3/3.5 gave lots of incentives for fighters with at least 13 Int with feats like Expertise, Disarm, etc.
Same issues + others with that kind of method. Preqreqs based on stats totally spit in the face of what i'm trying to do. I don't want int fighters fighting just as well as strength fighters.
Besides it still pushes the same cookie cutter optimization methods for characters - albeit maybe the cookie cutter fighter needs 13 int in this game to pick up X and Y feats to be the most optimized. That's what I'm trying to avoid.
In this proposed system, however, what incentive would a fighter have for raising his Int score to 18 or 20? If the only reason is because he is no longer able to invest in the stat that truly helps his character perform in his chosen role, that hardly seems fair to the player who chose that character in the first place.
When you start arguing fair to the player I tune out. That's a boring and simply untrue argument than can be made about anything ever for all time. I mean what the heck does fair to the player even mean?
Anyways, the whole point is for players to play characters they wouldn't normally play because traditionally they aren't optimized enough. I'd say this accomplishes that with flying colors. I've still not seen an example of it not doing so.
The game is supposed to be about choices. If a player chooses to play a fighter who doesn't have the best strength, that is their choice. They can do that themselves.
In either case there will be plenty of choices
They don't do that now - and why would they? It results in an inferior character. So my proposal is to keep strength the most desirable stat for a fighter but to free the player from the cookie cutter optimization that is currently in the game and make optimizing each character an interesting endeavor.
Having the dice or DM decide it for them is having someone else make the choice for you. I'm sure there are better ways to accomplish this, but D&D has never been very good at it, especially in early editions which unfortunately set the standards with similarly flawed logic.
My problem with rolling stats for character generation is that it makes you worse for the whole game with no hope of changing. With random stat caps you at least start out on equal footing and can adjust your character as needed as you level to take into account those discreapencies.
All that said, if everyone in your group is onboard, go for it. Its no worse than rolling stats randomly to determine what you'll be playing for the rest of the campaign!
My only issue with random stat generation is power discrepancy can be really high. random caps with point buy generation doesn't cause those big power discrepancies and can be planned for by the player in the leveling process.