A gaming book typically needs to be built like a reference book. And small paperbacks are not good for referencing. Look at most of the textbooks you've used in school. They're typically use large pages for the same reasons that gaming books are large. They are easier to use that way.
You might think that it would be easier to flip through a smaller book, but this isn't actually true. Let's turn the PHB into a standard paperback - the pages are now much smaller, so there are a lot more of them. In order to cut down the page count, art is eliminated and layout is simplified.
What you now have is a book with more pages and fewer landmarks. This makes it harder to flip through to find any particular piece of information.
You might think that it would be easier to flip through a smaller book, but this isn't actually true. Let's turn the PHB into a standard paperback - the pages are now much smaller, so there are a lot more of them. In order to cut down the page count, art is eliminated and layout is simplified.
What you now have is a book with more pages and fewer landmarks. This makes it harder to flip through to find any particular piece of information.