Like I said, then you might want to start looking at another game with less prep time.
As for me I tend to run pretty streamlined combats as well. I find that one of things that slows down combat is players not knowing what thier own abilities and spells do. But I try to cut that down by using spell cards and having them jot down what thier feats do on index cards. As for prep time I'm sure youre busy as am I. I work full time and on off nights while my wife is in class at grad school I take care of my 4 year old son. Not to mention the typical day of child rearing is busy at best. I still find the time to prep for both a Mutants and Masterminds game and a D&D game that I run.
Even if I prep for 4 hours and the the PC's run through whatever I put in thier way (and lets be honest depending on your players there's always a chance of that happening, whether you prep for 1 hour or 8), it's still worth it to actually be prepared. I'm of the school of 'better to have it and not need it , than to need it and not have it'. As for rather focusing on characterization and plot depending on the type of game youre running those might just be more important, but then that's a choice, your choice, on what to focus on. I like a balance and honestly plot and characterization are easier and take less time for me to nail down than stat crunching.