BenjaminPey
Explorer
And the party inside sees the goblins doing this and reaches outside the hut to brush them away, or just, I don't know, steps outside to kill the goblins? Only the caster has to stay inside for the duration, and it doesn't obscure their vision.
But if you fight the goblins before resting, there's no problem, is there?