It's fractionally faster when you're just walking across the board. It's incredibly faster when you're going to move, think about attacking, look up a rule, decide not to attack that target, attack a different target, answer the door because the pizza's here and you're buying, then try to remember how far you've moved so you can move again. The increase in speed approaches infinity if that rule lookup turns into a rules debate and by the time you're done two different people remember you starting in two different squares.
It's never gotten quite that bad for us, but close. We did it for years in previous editions, and we'll do it again when another game calls for it. But there's no way I'd house rule going back to it in 4e. It's too fun and there's too much stuff tied to it that would have to be ignored or changed.