Well, technically you're describing 30+ enemies vs. 5-6 defenders, and I'm guessing it will occur within a day. It's not a siege, it's King of the Hill. Essentially it's a complex encounter template (to use 4e speak).
About your concerns regarding overwhelming the PCs, I think you don't have to worry. If you have 20 level 5 minions (50 XP), that's a normal 5th level encounter for 5 PCs right there - add a leader and a hazard and you've got a real challenge. After all, the PCs have terrain advantage with lots of cover and (i'm guessing) higher ground.
With PCs outnumbered 4 to 1, those are pretty standard fantasy adventure odds, and you can expect the PCs to hack through them, especially if they have AoE powers.
Some things to consider...
* Do the PCs have time to set up defenses? Allow them to change some of the cover around, establish arrow slits, barricade doors, boil lamp oil to throw on attackers, cast a ritual or two, or even create quick traps (see Madmartigan in
Willow).
* Do the monsters need to get to them quickly? If not, there's all kind of things they can do: Starve the PCs out then attack while they are weak. Call for reinforcements or perform a ritual. Set demolition charges. Make circle of oil + flammables around ruins then shoot flaming arrows & smoke sticks into the ruins, smoking PCs out into the now burning circle around them thus pinning them down for archers. Create simple siege weapons like a mini-mantlet, hoist, or battering ram. Throw grappling hooks to a damaged wall with the ropes attached to a team of oxen and have them topple the wall. Create a tower shield wall around the battering ram. Trebuchet goblins over the wall...

Just kidding.
