As others have noted, use non-damaging environmental effects to challenge them.
Cold: Slippery Floors, fog, sleet storm, Thin Ice over running water (a killer with a couple of giants on the other side).
Fire: Molten Lava, poison fumes from said lava, lack of oxygen in some areas.
Also, you...