In my experience, optimizers relish the thrill of the hunt away-from-the-table and want to show off their finds at the table. The trouble is being a DM at a table with optimizers. There seems to be one of four possible approaches to dealing with an optimized character and an optimizing player. First, you outright ban optimization. Second, you ramp up the combat challenges to such a degree that the optimized character is properly challenged...which will almost guarantee the non-optimized characters die regularly. Third, just never feature combat. Fourth, do nothing and let the optimized characters constantly walk all over any and all combat challenges.
None of these solutions are particularly great. Banning optimization cuts out a chunk of fun for an apparently significant segment of the gaming population. Ramping up combat challenges grinds through the non-optimized characters and basically forces them to optimize or die. This is an especially bad solution given that a not insignificant segment of the gaming population does not care to optimize, so essentially forcing them to is bad. Never having combat kinda defeats a major part of the fun of D&D...having tense combats. And letting the optimized characters always trivially defeat any combat challenges also defeats a major part of the fun of D&D...having tense combats.
I'll share my approach, which is very much coming from the "we can all get along and have fun together, despite our differences" school of thought. Of course, I don't know your group, so YMMV.
1. Selectively Banning of Things in Advance
If you're sufficiently familiar with the rules and know the sort of campaign you want to run, you can have conversations with your group in advance about what
specifically is banned for your campaign.
For instance. I'm running a game based on ancient Egypt with overland travel being a significant challenge.
Leomund's Tiny Hut is not available in this game (in fact none of the traditionally "named" D&D spells are available), and when a player wanted to use a Genie Warlock (which was off my list of allowed subclasses), I read through it and said everything was fine except for the 10th Sanctuary Vessel allowing the party to have an extradimensional resting space not unlike tiny hut. We agreed that if he got to 10th level as a warlock we'd swap that feature out.
2. Compartmentalized Escalation of Combat Difficulty
There's this idea about combat – perhaps reinforced by our tendency to fall into "boardgame think" when using maps/minis/VTTs – that the players line up their guys, the DM lines up the monsters, initiative is rolled, and things proceed in an orderly manner with no surprises until one side or the other hits 0 hit points. The more you break out of that model, IME, the more satisfying and engaging combat becomes.... and the more you break out of that model, the easier it becomes to present multi-layered challenges designed for optimizers and non-optimizers alike.
Here's an example (I hate working with hypotheticals, but I don't have an example springing to mind from my own games recently): Your optimizers are a ranger and a warlock who favor stealth ranged attacks, preferably with an alpha strike. Your other 3 PCs are non-optimizers who feel left behind when combat breaks out. The encounter hobgoblins in a canyon. To challenge both optimizers & non-optimizers, you might include the following elements to the encounter:
- The hobgoblin caravan is transporting prisoners, one of whom is a high-value official with info the PCs need, but the official cannot be positively ID'ed from far away. This is a task for the non-optimized PCs.
- The hobgoblin caravan has outriders – scouts mounted on draft horses with blood hawks bonded to them via beast sense. If a bloodhawk screeches or an outrider sounds a horn, the hobgoblins go on high alert and some of the prisoners could be threatened. This is a complication both groups of PCs will need to cooperate to workaround/evade.
- There is an awesome ledge for sniper PCs to perch on up ahead, where a bugbear wielding a crossbow and poisoned bolts had positioned itself. This is a challenge for the warlock & ranger optimizers that can then turn into an opportunity.
3. Challenge Their Non-Combat Abilities/Strategy/Teamwork/Creative Thinking
So much fun in a D&D session can come from thinking laterally, thinking outside the box, and otherwise resolving situations that may have nothing to do with combat. Chases, investigations, moral dilemmas, court cases, masquerade ball intrigues, infiltration scenarios, etc. It's not about going to the extreme of "never having combat" (as you implied), but providing a diversity of challenges besides just combat.
For instance, there was an (optional) tomb my party was exploring which had been recently delved. There were a few traps and at the end a few mummies, but the real challenge was figuring out the story of what transpired in the tomb when a party of NPCs delved within. It was essentially an investigation mixed with some navigational problem-solving and a hieroglyph puzzle. One thing that stood out was a fallen stone block with a narrow crack/fissure running through it – a trap triggered when the NPC party left the tomb. PCs sent a cat familiar through and a PC with
misty step, narrowly surviving a trap on the other side which revealed a leverage point in a hidden chamber above, then the dwarf PC identified there was a slight downgrade slope, PCs poured water and oil (there were channels in the walls which held lamp oil as a form of ancient light source) to lubricate the stone block's contact with floor, and then a lot of elbow grease got it moving. All my players stayed engaged, including the optimizer.
4. Let the Optimizers Shine - Judiciously "Do Nothing"
Again, you seem to be leaning toward extremes. Just as I've included dramatic secret meetings with a dowager queen for an Actor Player running a Bard PC, when I have an Optimizer Player I want to deliberately provide moments in the game that speak to their play style, spotlight them showing off their cool combos, and let them kick butt and take names.
There can be a formulaic approach to planning D&D combat that actually becomes the enemy of holistic design appealing to multiple play styles. Not every combat should be designed the same way. Sometimes it's great fun to trounce a lone monster in one turn. Sometimes it's fun to have a big multi-layered set piece encounter firing on all cylinders. Sometimes it's fun to shake up expectations and either hard or soft counter specific PC abilities to require creative thinking.
