Two perfectly plausible backstories:
1) The Chieftain of the Hobgoblins (or, even better, the minotaur!) is amassing humanoids in the forest slowly and quietly to launch a large-scale attack on the nearby keep, to take it, loot it, and leave. The coalitions are still being forged, so each tribe has its own cave system, but alliances are happening, and some creatures are working together.
2) The Humanoids have gathered, but under the superstition of the humans that the caves are haunted. They are banded together for strength in numbers, even if they have their own communities.
I really don't see the need for any explanation than that. Keep in mind that most of the humans in Basic D&D were NPC 0-levels - against a bunch of 1+ hit dice humanoids, and scores of 1 HD humanoids? There was a NEED for heroes to raid the place.