That'd totally work for me.What if you just gave PCs all the benefits of leveling up except for the hit points (assuming they start with some decent chunk of HP)?
I actually was recently fiddling with a hack that would work much like that. A character would start with his Con score in hit points and then gain from 1 ( wizard) to 4 (barbarian) hit points for each Keven ( including the first). It improves a character's hit points for tier 1, and then stays pretty static for tier 2.
It had the neat effect of working well ( in theory at least) with the Encounter building guidelines, CR and such, when it comes to the damage a PC can take. The numbers line up will enough that I'd just treat the characters as equivalent to half their level. So, level 5 characters would be facing CR 2 or 3 creatures as the standard.
On top of this, the players would still have fireball and their Extra Attack, which effectively means the monster's hit points have been reduced, too.