Okay, let's start from the basics. Most monsters are, in 3.X, balanced against a group of the 'core four'. Groups of six usually have more rarefied characters who might not be used if those bases weren't covered....and they tend to have a stacking effect. As you go up in levels (and I've run a group of six from 1st to 28th), you're going to discover that some parts of 3.X need a shift in thinking to continue.
You have just hit the 10th-level shift. Players now have access to spells and abilities that make things much different than before. The nature of combat has now changed and your tactics as a DM to challenge players will have to change, as well. Your players are now going to begin teleporting, scrying and blasting in ways that will confound you. There have been a bunch of good threads concerning higher level games here that you can find by searching, but here's some pointers.
1) "
The Piratecat Method": He's not just the president, he's a member.

One of Piratecat's best tricks is to take a popular monster, change his stats every so slightly and then resell it as an entirely different creature with a new set of paint. This requries very little DM time to accomplish and can reap great results. For example: at one point, PC's players came face to face with a huge undead construct formed entirely from writhing zombies interlocking in a mass that formed a giant worm. It had a 'head' of sorts, where arms formed a mouth with daggers for teeth. In actuality, it was a slightly modified Purple Worm, but the players didn't know that...and they had an exciting and tense encounter. And when in doubt, use the 'speed advance' option. Advance a monster by giving it a few levels; those bone devils are CR9...barely a speed bump in a straight up fight: make them CR 12s just by giving them an additional 3 HD, raise all their d20 rolls and DCs by an extra 3 and call it a day.
2)
"Everybody Rolls a ONE" You need to accept that NOTHING you throw at your players is invulnerable. And that's OK. Your players fought hard for those skills...let them flex them. I once sent a Winter Wight (CR22) against my party, who were 17-18th level at the time. Twelve rounds in and they'd barely scratched it. It withstood over 25 spells from the three spell-casters, healed damage as fast as it was dealt and was overall very cocky and mocking as it slapped them around. As they prepared to flee, hoping some might survive, the wizard threw a disintegrate at the Winter Wight. The only way could it work is if the monster rolled a natural '1', which it did...and combat immediately ended. Such are the ways of high-level combat. Many battles can be decided one to three rounds.
3) "
A foe for every hero and a hero for every foe": The problem with a single monster (though they're great...sometimes) is that in a large group, some folks are going to have NO use fighting them. Around 11th level, those players I mentioned fought some Frost Giants from a frozen path in the mountains. This was great for the archers and the wizard....but the cleric and the rogue went to the back of the cave and cooked up soup....
because they had no way to contribute. The environment didn't play to their strengths and the foes weren't ones they could fight in that situation. It sounds like your players are facing a similar problem. The problem isn't victory that's got them down, it's boredom. And I'll bet it's because not only is there isn't a challenge, it's because often the battle is only relevant to
some players. Two bone-devils and a 14th level fighter are all well and good, but a fighter could defeat them all in the right conditions. You might have fared better with Under what conditions? If the answer is: in a room that was X' by X', that's possibly part of the problem. There's a reason that 3.x separated Encounter Level from Challenge Rating. Four vrocks are more than four times as dangerous as ONE vrock...because of their interaction, they make for a tougher encounter. Two shocker Lizards are more dangerous in a wet environment than four are in a dry cave.
4) "
Spoilers can be tougher than fighters" Those bone devils, even at CR 9s, should have been able to really mess the party up with one spell alone: Wall of Ice. I mention this not to say 'learn better tactics' (
because realistically who has time to plan all of that out for every dang monster ) but to change the monster's goals. First thing I ever do with any monster is look at their powers...in this case, did the devils make things difficult or did the fight straight up? Did they just attack or did they use tactics like attacking defensively and their poison? Remember, they don't have to fight to win, just to wear the PCs down or keep them busy until the more powerful dude had his shot.
5) "
Diversify the Portfolio": Use monsters that the PCs don't know that well, like the Digester, the Ethereal Marauder or the Yrthak. Look into the DMG sections for weather and hostile terrain. You'd be stunned how nasty an enemy deep cold or heavy winds actually are. Force your players to fight in places where they're at a disadvantage. Make them go in the water. It's HORRIBLE. If they prepare, let them enjoy the fruits of their labors. If they don't, let them find out what happens. Make them protect weaker allies against monsters, like townsfolk or nobles threatened by vampire spawn or something.
Hopefully that gives you some quick ideas. You want more? I got 'em.