You're not "losing" XP. You're "spending" them. You're investing part of your life's energy in the spell you cast or the item you create. A semantic difference, true, but a meaningful one in my opinion.
XP do not just represent experience and practice. They also represent personal power or life force (negative energy gives negative levels) and "hero status."
Typically, you will come out ahead, spending small amounts of XP to overcome challenges that result in you earning larger amounts of XP.