Alright, here is what I've decided on:
The coliseum is part of what was formerly the duergar empire. The Duergar have become slaves to the Elans (from the Psionics Handbook) and their cultists. The Elans are servants (also aberrations) of the mind flayers that once ruled the Duergar. The Duergar were formerly ruled by a group of especially crazy Derro. The Derro have been taken control of by the Elan telepaths, and now serve as tools for the manipulation of the kingdom.
Stone giants are also involved, they were formerly allied with the Duergar kingdom. The Elans will act as a sort of hook to relate all things to the mind flayers, while the Duergar will be the main obvious villain. So we have a stacked pile of manipulators pulling the cards.
The characters are all currently captive and being used for slave labor, but they are not guarded nearly well enough. Once they get past their mannacles and chains, it should be easy for them to overpower a guard or two and acquire some mundane weapons. Then I expect them to attempt to re-acquisition their gear. I'm going to try and keep 10-15 points of subduel damage on each character at a given time, so that they can be captured and thrown into the coliseum if they get into a mess they can't handle. If they do end up in the coliseum, I'm going to have a slave/gladiator stone-giant as the local champion, but he is only a gladiator because he KNOWS that the Derro are being manipulated by the Elans.
There should be plenty of opportunity for escape if they get outside of the city, but if they end up in the coliseum I've got a completely different plan for next session in which they will need to find their way out of the city (probably through an underground cavern, though I'm not "entirely" against them trying to really fight their way out if they are at 6th-7th level at that point, or even to bribe the guards). The group is currently 5th.