If the cave is full of Kobolds, then on day 3*, when you return, your face is full of Kobolds
*not necessarily on day 2, because they may figure that they drove you off on day 1, but when you return day 2 then leave again, they're going to know what's going on.
They have two options. Run away...