Let's look at an all martial party. A 20th level battle master fighter, samurai fighter, berserker barbarian, and assassin rogue. No magic items
.How do they take down a Demilich (CR18, which should be an easy encounter)? It's immune to non-magical weapons.
Easy - Holy Water, Oil and Torches available at every adventuring shop in the multiverse. The 4 you mention should have no problem downing it with little creativity, provided they have average or better abilities.
If your completely non-magic party didn't buy holy water, or oil, its a bit more difficult but it is not immune to grappled, does not have blindsight and its attack relies on seeing the enemy. Gapple it, cover it with a bag, light a fire and have your Raging Barbarian hold it in that fire until it is dead, you may have to switch out to one of the fighters due to the AOE legendary, but with 4 characters with roughly 700 hps between them, eventually you should be able to kill him if he doesn't escape and flee first.
Now these two answers assume you are playing in a game where you know you will be expected to fight monsters only damaged by magic B/P/S and you did not bother to take any races or feats that provide cantrips, elemental damage etc and also did not bother to take any skills or proficiencies for building magic items or did not have downtime that allowed you to make magic items.
The mummy rot from a CR15 mummy lord would be a death sentence if they can't find an NPC to cast remove curse on them (or simply can't reach them in time - no access to teleport).
Sure, if they fail the saving throw, which should be very, very unlikely for the fighters and unlikely for the Barbarian.
Again here - not immune to grapple and VULNERABLE to fire.
It is pretty simple - Prone-Grapple-Oil-Torch. He can use his legendary action to get out of the prone but then he can't use his stunning legendary if he keeps using it to beat the grapple.
It is 4 on 1 here. Clever use of initiative order wins this fight every time even with his legendary actions. The rotting death might take one of the characters a few days later as you mention, but a Wizard in the party is no guarantee against that as few of them have Remove Curse in their book IME, fewer still in a game without downtime to do things like build magic items.
If most of them are strength based, even CR 3 manticores could give them a hard time (because strength based ranged weapons have a pitiful range).
Since javelins only have a maximum 120' range, the manticores can move away after they attack and perpetually remain out of range, allowing only a readied action at disadvantage to counterattack. Admittedly, this is not nearly the same issue for dexterity oriented martials.*
You have an Assassin, who presumably is not using strength-based weapons. And even if this is a melee assassin who dumped dex to an 8, at 20th level she will still take down a Manticore with a Light Crossbow by herself
EASILY. Like 2-3 rounds easy without any help.
Strength-based Barbarians and Fighters are not limited to Javelins. All Barbs and Fighters are proficient in Longbows. These three have 11 attacks a round between them (4 Battlemaster, 5 Samaurai, 2 Barbarain) and that is without using action surge or frenzy. With a Longbow, those attacks have a +5 attack roll even if they dumped dex to 8 and they have a better range than Manticore spikes. Further the Battlemaster has maneuvers (trip attack on a flying manticore), and the DC for that maneuver is based on strength even if he is using a bow and should be 19 at this level.
With an 8 dex those characters need a 9 to hit a manticore. The chance of a manticore in flight surviving a single round against those foes is 26%. 3 out of 4 times these four warriors, all optimised for melee and dumping dex are going to take out a flying manticore using ranged weapons in ONE ROUND. And that is without using action surge or considering fall damage from trip attack.
These tactics and gear I mention in the three examples above are all common mundane options in the game and defeat the strawman foes you provide relatively easily (the manticore especially easily). And that is after you really go out on a limb with this build as these characters have 25 ASIs/Feats between them. What did they use all ASIs those on?