When the DM is less tactically competent than the PCs - or doesn't know how to exploit the rules as well - these sorts of things happen. It's the unfortunate effect of not writing out optimized combat behavior in the monster entry (I don't mean there are no combat routines - I mean the ones offered usually suck.) I don't mean any offense by this, but your remark that AC 30 is "impossibly high" at 10th level indicates that you are not an optimizer. (An AC in the 50s can be attained at 6th level, if you optimize.)
In general, when you DM for powergamers, you need to start thinking like an optimizer, if only to appreciate the angle they come at things from. For example, what do you think class levels are? If you answered that they explain a character's identity, you've given a predictable non-optimizer response. The optimizer thinks of levels as the
currency with which he buys feats, skills, and class abilities - those things that make up the engine of destruction that is an optimized character.
You, as a DM, need to think about things that way too. Only, you aren't buying levels and LA, you're buying HD and CR. The easiest thing to do if you have no particular goal in mind is just hunt for "bargains." Many templates cost half as much to put on a monster or NPC as they cost in LA - and many are monster only. A PC becoming a vampire is making a losing trade - but an NPC gains the exact same benefits at the cost of only a slight increase in CR. This is an example of what you might consider a bargain.
Retreater said:
Here's what's been going on. This isn't based only on the number of players we have (though there are a lot). Many enemies (of equal CR to the party's level) have to roll a natural 20 just to hit the characters. Other enemies, even with a load of hit points, pretty good AC, and DR are getting chopped down left and right. The worst thing though is summoning. Both the druid and the cleric are throwing lots of monsters/animals with many attacks. It's slowing down the game by adding even more combatants into the fray. It's not unusual to have a 2 hour combat with what amounts to 10 or more PCs (including their summoned creatures and animal companions).
There is a spell called Wraithstrike. Use it. An abnormal troll with 4 levels of sorc (non-class, so they count 2 to 1 for CR purposes, meaning you can get the levels for 2 CR) is much more of a legitimate melee threat than a troll out of the box. He also has better will saves, making it less likely that a party will just yawn and dominate/sleep him.
Alternatively, throw casters at the enemies that can ignore the summoned creatures long enough to lob sudden maximized fireballs, then flee. Villain escalation is a perfectly normal part of heroic stories, but you should be careful to depict it as such, and not as the DM's private revenge.
So my problems are dealing with summoning, dealing with impossibly high ACs (in the 30s at 10th level), spell effects that (when combined through 2-3 casters) inflict close to 100 points of damage/round, and having monsters that can do some significant damage to a character before getting turned into ground beef by summoned creatures.
Retreater
For summoned monsters, dispel magic is the way to go. Try babaus - they get it at will.
It sounds like the encounters you've been throwing are too generic.
Have you tried throwing incorporeal undead at the party? Don't alert them to their presence, just have them wake up and start taking stat damage in the middle of the night, with no visible source. The undead remain underground, attacking from cover, and do this incessantly. Sure, they can levitate or whatever, but how many days can they keep it up for before exhaustion sets in? Their enemies never tire, never sleep, and return each time they are chased away. (Obviously this encounter needs to take place many days' travel from towns or aid, or it will be meaningless.)
Have you tried challenging them with numbers? A single level 1 battle sorcerer or warmage is EL 1. Using the doubling rules, x2 = +2 EL, so a level 11 encounter can have 16 level 1 mages. Surround the party, give them all improved initiative, and have them cast magic missile on the same target, with a few reserving missiles to disrupt the party's spellcasting.
While you do all this, though, always keep in mind a few basic rules. Often characters win because the players are privy to knowledge that they should not be. Enforce metagaming rules strictly. The
players know that demons are easier to kill with cold iron, but the characters don't know that unless they have the knowledge ranks. The
players might know that whatever they face has an average AC of X and hit points from Y-Z, but the characters don't know that.
Of course this goes both ways - you shouldn't have an enemy that uses precisely the right amount of force needed to incapacitate the PCs, because the enemy has no idea. An enemy usually veers towards overkill because he wants to be certain the PCs are dead.