Spells are supposed to be powerful, aren't they?
Fireball doing 8d6 at 5th level is huge damage. That's 28 points of damage on a failed save, 14 on a made one (on the average). Level 5 PCs likely have 27-55 hit points if they are full. That's a huge amount of mass damage.
Fireball has always been a big hitter; dropping one of these
should be a serious threat to a 5th-level party. Similarly, it should also represent a significant expenditure of resources to a 5th-level wizard, and it does. At 5th level, a wizard can only cast this twice per day.
Sleep (AFAICT) allows no save and goes off of current hit points. After some softening up, that spell can reasonably expect to take out 2-4 party members if cast at an equiv. level to the party.
Sleep gets weaker as the party levels up, just as it did in previous editions. At 1st level, it's pretty amazing, but even by 5th level you'll be lucky to have it affect more than one same-level enemy; casting it with a higher level spell-slot will provide sharply diminishing returns.
Armour of agathys is just crazy. Say the spell is cast as a 4th level spell. And say the attacker is doing 15 points of damage per hit. In two attacks the attacker will do ~8 points of real damage and have taken 40. And a warlock can cast this after every short rest (it lasts an hour).
Given that casters have cantrips and so can do quite a bit each round (a lot with some work), I'm just a bit worried that the caster "nova" is too much. I also think (as in 1e-3e) caster opponents are going to be scary as heck.
Unfortunately I don't have my PH with me, so I don't know offhand what
armor of Agathys does, but doesn't it depend on the warlock being damaged in melee? I have no doubt the spell is powerful, but if I'm remembering correctly, that's a huge limitation.
At any rate, casters are likely to be more powerful in 5E than non-casters; that's part of D&D's history, and part of why pretty much every class has magical options. Fortunately still, 5E was designed with clever limitations: bounded accuracy on stats, severely reduced numbers of high-level spell slots per day, and a hard limit on buffs thanks to concentration rules. Overall, I don't think you should worry.
The level 11 party Sorcercer is throwing out fireball after fireball of insane damage. The Rogue, when getting a sneak attack is hitting for high 20s, and can crit for 40-50 which the fireball can do to multiple targets all at once.
But the rogue can do that all day (and attack with advantage, too) potentially without being noticed, while the sorcerer is eventually bound to run out of spell slots and spell points all while attracting the attention of everyone who sees him.
If you're finding that your sorcerer is nova-ing too much, try putting more pressure on the party so that the casters aren't always able to stay fully rested.
I have a Cleric/Wizard NPC (Knowledge cleric), hitting with Sacred Flame for 20s. A level 6 Magic missile hits for 29 automatically. Inflict wounds level 6, 8d10 damage????
Casting fireball with higher-level spell slots (like in the example above) is worse damage-wise than just using that slot to cast a higher-level damage spell, but using a 6th-level slot to cast
magic missile or
inflict wounds is honestly a huge waste. Yep, if you're willing to burn your only 6th-level spell slot to cast a
super-duper magic missile, it ought to be better than a cantrip, but the numbers you provided show that it's only barely better even including the fact that it auto-hits.
I am trying to make more encounters last longer, but unless I legitimately throw 4+ medium encounters in any day that I have any encounters at all then the non casters are just not going to compete. I just don't see casters running out of slots very quickly, especially if they sprinkle in cantrips which isn't much of a penalty when you can cantrip for nearly as much damage as physical damage dealers.
Cantrips are meant to stand-in for weapon attacks for spellcasters, so they ought to be roughly on-par with (or perhaps only slightly worse than) what non-casters can do with basic weapon attacks.
Overall, I definitely believe you should put more encounters on the party during each day. For example, if the party is crawling through the Redbrand hideout (Lost Mine of Phandelver), they shouldn't be able to take more than one short rest in that whole dungeon before the BBEG escapes and all the enemies start scouring the place for the party that's infiltrated them; if the party does any fighting down there and then leaves for a long rest, when they return the base should be either abandoned (and most of the treasure already taken) or completely fortified against another incursion. This should be true for almost any dungeon on intelligent creatures.