"Temporary hit points aren't real hit points. They're a layer of insulation that attacks have to get through before they start doing damage to you." (PHB, page 293)
I.e., losing temporary hit points doesn't harm you, it only wears down your stamina reserves.