The loss of potency is nowhere near as large as you imply for casters. Yes, it is reduced (especially compared to 3e), but it absolutely is not nearly that dramatic.
Casters got significantly nerfed in 5e. Justifiably I will add. They were ridiculous.
Let's start simple with magic missile. Od&d casters got 1 missile at first but autoscale, 5e get 3 and only 3 without upcasting. Let's assume the 5e will upcast through career....
Let's consider a prime magic missile target: kobolds.
Kobolds in 5e have 5hp (2d6-2) vs ODD 2.5hp (1/2 d8)
Each od&d magic missile can slay a kobold while it takes two 5e missiles. So at 1st level, 5e slays 1 & injures another while the ODD slays one. Then at 6th level the ODD slays 3 while an upcast-as-3rd slays 2 and injures a 3rd. At 11th ODD slays 5 while upcast-as-5th slays 3 and injures a 4th.
So even upcasting, the 5e caster is falling behind.
Let's take a common fireball target example: troll. A 1e & 2e troll was 33hp (hd6+6), 3e was 68hp (6d8+36) and a 5e troll is 84hp (8d10 + 40).
That 1e fireball is ~50% of hp, the 5e fireball is 33%. At 10th level that 1e fireball is roughly 101% of trolls hp, so dead. The 5e fireball, upcast to a 5th slot is 41%. Go to a 5e 9th slot for giggles and its 60% of a troll.
Lets go back to the magic missile, which is 14% of an 1e troll at 1st level or 12% of a 5e troll with a 1st level caster.
Now scale up. 6th level ODD caster does ~13hp (40%) while a 6th level 5e caster using a 3rd level spell slot does 17hp (20%). Up to 11th level, ODD caster ~22hp (66%) and 5e caster using a 5th slot does 24hp (30%)
16th level? ODD 32hp/95% vs 5e 35hp/40% with an 8th level slot. Well, probably not as there's likely something better but still, scaling example of an ODD caster essentially slaying an uninjured troll with a 1st level spell slot which is simply un-possible in 5e.
Then there's concentration. In earlier editions, spells layered and persisted. You want to use up your spell slots casting 4 buffs and speed run a dungeon? Go for it. Not possible in 5e because of concentration.
Let's talk save differences. In earlier editions fail once and be affected the whole time. Most 5e spells are save every round. An ODD caster could Hold Monster and just completely ignore that creature until everything else was mopped up. 5e that buys you a round or two to set up a group auto-crit.
Assuming it doesn't have a Legendary to nope out of Hold Monster entirely.
And then high level spell slots. A 20th level 3e caster had 4-6x 9th level spells/day, and similar 6th, 7th and 8th. In 1e & 2e, a 20th mage got 2x 9th and up to 4x 6th level. Meanwhile 5e characters get ONE of each 8th and 9th spell slots and no more than 2x 6th & 7th.
That means after burning half their upper level spell slots, an earlier edition casters still retains had more high level options and unlike a 5e caster, didn't need to upcast any spells to deal meaningful damage (i.e. kill a troll with a 1st level spell).
And yes, 5e casters don't have to pick spells for their slots, but at higher levels where you have a finite number of prepared spells, exactly how many of a 20th level caster's ~25 prepared spells will be allocated to higher level spells where they have only 1 or 2 slots? If a 1e caster can prepare and cast both Wish and Meteor Swarm while a 5e caster prepares both Wish and Meteor Swarm but can only cast one of them then the 5e caster has
less flexibility. Sure, they can upcast, but the 1e casters don't need to because everything upcasts for free.
Which is actually why high level casters would hold spells despite having a number of them, they were too good to waste. 1e fighters actually carried most fights so the mage-zooka could nuke a balrog or a troll village or whatever. 5e cantrips let the casters contribute a bit every round.
So again, 5e casters got significantly nerfed. Justifiably. They were ridiculous in earlier editions. The spells are de-valued and there's the illusion of rapid recovery to encourage people to use them more with the knowledge they have cantrips to fall back on.
Fighters in 5e at least got all attacks at full bonus, some self-healing, a mini-haste, core class abilities for AC/blindsight/blocking. Subclasses provide bonus damage, movement, crit chances or spells without having to multiclass and lose attacks. plus 5.5e adds weapon mastery maneuvers.
5e paladins are just glittery fun smite machines with an eterna-buff, don't haven't be totally straight laced, always welcome, no notes.
Warriors also have to deal with hp inflation but at least the notably less powerful mages now have enough hp you can leave them alone for 12 seconds to go fight that ogre. (1e magic users might have 50hp at 20th level)