Jester David
Hero
It's tricky to compare. Things are very different now than in the mid-80s. The Basic Rules and ease of ordering books online help.You know, that's a really important bit of data for marketing purposes. The Red Box is one of the most iconic D&D products ever, and introduced a lot of people to the game. That's definitely something to consider in any analysis of what works or doesn't work. (I'm pretty sure they've done that analysis, hence the Starter Set. It would be interesting to find out how well the Starter Set has been selling.)
I still think the best way to do the Realms is just to throw some support to FR Wiki, and write some text for that.What they can (and I think should) do, is cover multiple eras in the book. For instance, when covering Waterdeep, they give you the history up to 1358, then talk about the important people and events then, and then continue the history, pausing for more snapshots of the city at important points (I'd recommend that they coincide with the first presentation of each edition), and after giving a snapshot of the current 5e stuff, they give future possibilities, maybe three or four different paths that Waterdeep could take. And they do that for all of the other regions of course.
I think they could easily fit all this into a book without crunch, and considering the very concise prose of 5e products.
A big book is nice, but it doesn't have search capabilities. And the larger the book, the harder it is to actually use...