The issue is that it doesn't need to be big to be either D&D or a complete game, and we know this because it has been in the past.
That doesn't actually follow - because D&D has changed over time. There are things now in the game that aren't just bloat and repetition that were not in the game before, but little has been removed. Thus, you should not expect to be able to condense it to similar low pagecount.
Again, this thread stemmed from me looking at the Pathfinder Core book and being filled with dispair. It's just too damn big. Finding anything in there is going to be a pain. lugging it to the game is going to be a pain. Passing it around the table is going to be a pain. Comparatively, my AD&D PHB is light as a feather.
The Pathfinder book (575 pgs) is effectively the 3.5 PHB and DMG (total about 630 pgs) rolled into one - with a comparative savings of $10 and 55 pages) to the consumer. The Pathfinder book is effectively smaller than the 3.5e equivalents.
Now, you may argue that the players don't need the DMG content, and I'd have to agree with you, but that's a bit of a different issue.
It seems -- and I haven't combed through, section by section, word by word -- to come not from an growing complexity of the rules, but rather a bloating of the text itself, an expansion of the language used to convey the rules and the layout of the book. Can it be nicer to read on the can, or easier on the eyes? Sure, but its utility as a rule book is lessened and its inconvenience in play is increased.
Well, this brings up a large question - what use is more important? Learning or reference? The extra verbiage is important for the former - a certain brevity is good for the latter. I submit that the design of the book is a compromise, and that's kind of necessary from a business standpoint.