You would need to come up with a system for scaling damage by spell level, irrespective of a spell's other effects.
You get to spend a lot of time defining healing effects under a system that doesn't like them very much.
PCs are likely to be, in general, more resilient and able to bounce back, but in the long term, you can forget about a 20-level campaign unless you introduce Conviction/Hero/Action points. Damage saves take people out.
On a related note, you have to decide on deadly DCs versus non. If you allow deadly attacks, a good roll could kill anyone in one go. If not, then being chewed in half by a Gargantuan dragon is potentially survivable. Different d20-based games have set different defaults, whether an unharmed character can end up, at most, dead, dying, or unconscious, after a nasty damage save.
Obviously, Favored Class bonuses will have to be restructured.
If Toughness steadily advances, you end up with nearly invulnerable PCs, even without strong defenses. If it doesn't, the genre conventions will be affected, as even the most powerful heroes will think twice before going toe to toe with something like a giant or a hydra.