Keep in mind that a fighter with 32HP and 10 surges doesn't have 32 life, she has 112 life, but only access to 32 of it at a time. Think of running. You could probably run for more than an hour a day, but only for 10 minutes at a time.
In my fights, bloodied is what happens when you get hit ONCE, one bludgeon, one gash etc. Any hit points you lose before bloodied are minor scrapes/bruises, lucky or exhausting dodges, all the things you can imagine a player uses to mitigate damage. As the fight goes on you get tired, use up your luck, your adrenaline wears off, this is damage. You get two major hits before you are down (bloodied, then dying).
I would not give out penalties for bloodied, since this heavily nerfs any build that uses bloodied to its advantage. You could implement an injury system for characters that are dying, or if they actually die in a combat (after failing three death saves, they are out of commission for at least an extended rest, and they sustain an injury, instead of death).
Remember that some players do not like injuries, it feels like their characters potential has been permanently lowered!
Either make the injury temporary, give them a way to get rid of it, or get them to agree to accepting the injury (for example, instead of dieing.)