As a DM, for monsters, I start at 0 and add damage. As a player, for my character, I count down from my max hp. I can't really explain why each feels natural, it just does - perhaps it just feels easier that way for characters, since they have much more healing, and hp going back and forth, etc.
However, funny story!
In a game some friends were running, one of the players was playing a big fighter type, really focused on tearing enemies apart, wades into the thick of things and so forth. And as the combat goes on, he's got a column on his character sheet where he is tracking damage.
And at some point, the DM realizes that the character has taken a lot of damage, and is wondering if he should have dropped by now. And he looks at the column on the character's sheet, and starts trying to figure it out. Is he counting down from his total hp? No, the damage is adding up... but it comes out to a huge total, hundreds of damage, way more hp than the character should have. And the DM asks about this, and the player answers:
"Oh, I haven't been tracking the damage I've taken. I've just been tracking the damage I've dealt."