As many others wrote in this and other Posts, put minions.
If you do not for plot / style reasons: Use insane amounts of hitpoints, legendary saves (so no oneshotting / disabling), high AC (full plate +1, ring of protection, buff spells, stoneskin potions/ displacement /invisibility whatever), or get the Party into disadvantage somehow. Everything is possible it is not that hard if you put a bit of thinking into the specific Situation.
I had a AC21 fighter with 20 Str and 150 HP with + 9 to hit vs. the 4 man Level 5 Party lately and it occupied them well, especially since they could not go all out in the Situation at Hand.
I second this post and just want to add that adding minions really adds to the fight for more characters as well. Martial characters tend to be very good at single-target damage while casters can really do big swathing hits on several minions. As well as having a lot of battlefield control and disabling abilities.
So, in short I find that it will be more fun and engaging for more parts of the party.