[4e] looks like a completely different animal with little in common with its predecessors.
I agree. It's got the same title, and borrows some title. But it's a completely different game. I like most of it. I especially like their online presence, as imperfect as it is. The updates are good and reliable. Most of the errors and the unbalanced characters are hammered down. It maybe the RPG that I'm least tempted to houserule. With AD&D, I only came back to a btb game in the last few years, and even then, despite my efforts, some houserules were necessary. But that's not the real subject here.
Back to the name issue. I'm not sure it could have been done without the draw of D&D's trade name. The books may have been able to be scaled down to a smaller production, but the service and support could not have. It would be non-existent without the D&D brand name.
Would it have been possible to create a new version of D&D that was closer to the original? I imagine many would answer in the affirmative, but gettin these very proponents to agree on the details would either have been very difficult, or would have yielded something almost identical to any of the retro-clones that populate the internet. Personally, I like Basic Fantasy RPG a lot. But I'm glad I have both that and 4e.
Be aware the system reads vastly different than it plays. It reads awful.
... the player stuff tends to read like an instruction manual.
Jester is right. The player's handbooks were written as reference manuals. They contain almost all the rules necessary for play. All the other books contain inspiration for the DM. There was no easy introduction for new players. To wizbro's credit, they gave a lot of support to local gamestores and the DMs that play there. And many say that a new publication will assist easing folks into the game. This doesn't equate to gygaxian prose, which, along with the novelty, was the draw for most of us back in the day. It's a more analytical process. Again, I have to say that I like the idea of writing the players handbooks as reference manuals, but the first one is more than a little outdated.
Personally, I'm not sure it's worth buying the rule-manuals anymore. I rely on the compendium, which costs about $60 a year. The DM stuff, I'm sure is still very useful.