Exactly. It's NOT the end all be all of everything.
The things that people tend to forget about casters is that your spells prepared and slots are limited. And if you're a spells known caster, forget about it. You're not preparing every great spell for every eventuality. If you prep the right one for the right situation, you're golden! Forcecage is a fantastic spell. I'm pretty sure I took it for my lore bard a few years back. But if you prep the wrong spells, you suck. You might as well have left a spell unprepared if you prepped Forcecage and you're facing a creature that's too big for it. Moreover, that's a 7th level spell so odds are fairly good you did not prep an alternate, so now your optimal use for that slot is to upcast a lower level spell.
Martials tend to be much more reliable in my experience. They don't really have either the highs or the lows of casters. A caster might end a fight with a single spell. Or they might just stand there throwing cantrips the whole time because they're functionally useless, having prepped the wrong spells. I've rarely seen an encounter where a martial is functionally useless. People often bring up flying monsters, but if you're a martial you should always have a ranged weapon. Less effective is not ineffective.
As for why those high CR monsters don't mess with martials as much as casters, I think it's because high CR monsters have more abilities to counter casters than martials. Teleporting is useful for escaping a Forcecage or closing on a squishy mage. But a lot of these monsters are melee based to begin with so there's rarely much point in teleporting away from the martials (unless it's to go after a squishy caster). This might legitimately indicate that casters are in fact stronger than martials, but unless you're homebrewing the counters out of high CR creatures, they're effectively not.