This biggest thing to remember is that no amount of changing how casters work is going to solve your issue... because the amount of magic in a game is 95% going to be represented by the characters in the party that have magic. So as long as you allow some form of casters in the game and then one or more players play casters... magic is going to seem really prevalent regardless.
Really the best way to do it is to do as has been suggested and only allow barbarians, fighters, and rogues... with maybe a monk as the closest thing to a magical person there is.
But let's be honest here... with the Battlemaster fighter subclass, you can have multiple fighters in the party and they all seem different. Same with the rogue. So it shouldn't feel weird at all with a party that consists of a Berzerker Barbarian, a Great Weapon heavily armored Battlemaster tank, a rapier and shield Battlemaster duelist, a archery focused Scout Rogue, a shortsword dual-wielding Thief Rogue, and an Open Hand Monk. They will all play together wonderfully and now you won't see magic almost never-- except for the times when you the DM decide to introduce it into the game. Which is how you would want it to be anyway.