You absolutely
can as a DM, lean into the Exploration and Interaction pillars. It just, more likely widens the martial/caster gap.
I mean,
Martial 
If anything, Combat is probably marginally-less imbalanced than the other two of the three pillars, since it's the focus of the least versatile classes, and DPR is notoriously easy to check up on.
In any pillar, casters can just learn/prep a variety of spells, and spontaneously cast the best one for a given situation, when the situation warrants using a slot.

In less important situations, you have cantrips to grind damage all day, skills (just like everyone else, likely more than some), non-combat cantrips, and, if time isn't the issue, rituals.
At least in the classic game, old-Vancian casting, and restrictions on the act of spellcasting itself, made that a bit of a challenge - heck, it made surviving 1st level a challenge...