But that's the point, we "baked in" the excitement by adding the rule. Before a natural 20 was exciting because it allowed you to hit AC's which would otherwise be "unhitable". Otherwise, a 20 is no different from a 10 as long as both rolls hit the AC.
I suppose if you want to have critical hits on 20, why not have minimum damage if you just hit the AC?
Others use critical hits but not critical fumbles, simply the 1 "always misses". So, why not use critical fumbles?
Just to be clear this is simply a discussion, I am not advocating one mechanic over the other.