I think it could definitely work, but the first question would be what such a setting would be about? Are the players just random humans who got sucked into the mushroom kingdom? Or can they also play mushroom citizens themselves (like Toad or Princess Peach), or members of the Koopa army? (maybe play as a Hammer Brother?).
Humor and a light-hearted mood would probably be important. I'm not sure if you'd even need hitpoints, since the Mario games rarely use them. Maybe something more akin to Mario Oddesey, where you have just a small number of hits you can take.
I like the idea that Super Mario 2 (Doki Doki Panic) used, of having a bunch of characters that each have their own specialty. Maybe when the players build their character, they can choose one such specialty. Such as jumping very high, very far, floating in the air, being able to hurl heavy objects very far, etc.