The spell slot idea is indeed a fine one! I've toyed with the idea of bringing back energy drain in the form of draining feature uses, be it spell slots or martial abilities... the concern I had was short rest features weren't too affected.
Then I realized that clay golems' max hp damage...