I think that, however, is part of the hindsigh. *Now* we know that the customer base has a major problem with change. It is not clear to me that WotC should have thought so at the time they had to make their business decisions. The customer base seemed to make the transition to 3e without major issue - heck, it grew pretty solidly! 3.5e was also fairly smooth.
It is the 21st century, the internet age. It wouldn't have surprised me if WotC (if only implicitly) thought, "Hey, this is an era where folks expect movement, expect change, and adapt!" Sure, there'd be some grumbling, but I am not sure they had solid reason to know the audience would be quite as reactionary as it was.
I think defining the customer base as "having issues with change" is unfair. As you mentioned the transitions before 4th edition all went very smoothly. The playbase certainly has its edition warring grognard who will scream it isn't Nth edition, therefore it sucks. However, up until 4th edition, people were mostly along for the ride.
A big part of the problem with 4th edition was how much it changed. Everything up until 4th was still clearly Dungeons and Dragons. 4th edition wasn't Dungeons and Dragons as every knew it. It changed massively. It was Dungeons and Dragons 2. It was a completely new game with a new flavor which utilized much of the Dungeons and Dragons setting and IP. It was a big enough change to make people stop and think "do I want what this new game offers?" Not everyone wanted it.
I think what 4th edition symbolizes more than people being afraid of change, is that the IP power of tabletop games is not as strong as some other mediums. With a movie you are only selling them on 2 hours, with a video game usually about 10 hours. With a tabletop RPG is a hobby that takes a huge amount of time up. So the D&D name will only carry you so far. People won't play something purely because it is labeled D&D. That might make them start playing it, but they will eventually gravitate towards what they like.
I guess part of the problem is the grognards always grumble in the past. The lamented the change from Thac0 to AC, but they always came along. So Wizards might have dismissed the grumbling about 4th edition as grognards being grognards.
4th edition being so different is a problem for wizards, because they are trying to shift back towards the feel of earlier versions of D&D. They now have a 3.5 and earlier playerbase and a 4th edition player base. One of those two will probably be unhappy with Next.
Really, thinking about it, maybe the best move would have been forking D&D. Turn 4th edition into spin-off (still called D&D, but with a tag, like D&D:Heroes) while D&D Next would just be "Dungeons and Dragons." Effectively have 4.5 and 5th edition existing simultaneously. If you have to outsource the custodianship of 4th edition to one of the better companies that writes 3rd party material.