After having read through a good bit of the replies, I've organized my thoughts a bit and thought I'd present them:
I think what bothers me about the 4E combat system "kludges" is not so much the use of a grid; I've been using gridded combat and minis since I got into D&D. Its the changes like reducing monster hit points by half, increasing character damage output, not using monsters above the character's level +2, using egg timers to limit character's turns and those sort of tricks to get combats down to a "reasonable length" that irk me. Overall, the RAW combat of 4E simply takes too long to resolve, grid or no grid. If I had to pull the same stunts in my 3.X game or Vampire game to get a combat under 30 minutes, I'd be highly annoyed.
QFT. I totally agree with you. This boils down the whole thread into one succinct post. 4E combat is slow by default. So slow, in fact, that the RAW game is nearly unplayable for many. This is a huge flaw of the system. It simply doesn't work for more casual gamers (or even hard core gamers) that don't have 6 or more hours to devote to a game session on a regular basis. In fact,the unreasonable length of 4E combat along with annoyance with neverending rules updates (PC nerfs) has seriously diminished LFR play in my area. There a some that want to play, but cannot devote the 6 or more hours to play an LFR mod. If the mods actually only took 4 hours to complete like they are supposed to, I am confident more people would be playing, but 4E's slow combat system makes the time investement too steep for many adults with busy lives. I think it's perfectly okay for a nice setpiece "boss" fight to last an hour and a half or more, but it is not acceptable to have every single fight that is supposed to be even remotely challenging to the PC's to take that long. A nice long, epic fight with the BBEG is fine, but I don't want to spend an hour or more fighting Bob the pig farmer and his four sons turned bandits or a bunch of kobolds. There should be an option in 4E's RAW to have quicker fights for non setpiece encounters.