Because those silent players are negating the enjoyment and benefits that non-silent players would be contributing in their stead.
There are limited seats at the table. Also, if only two people speak up most of the time, let's say, and the others are more seat-warmers, that gets tiresome hearing their voices and ideas all the time. I'd much more rather a democracy of relative equals than one in which 2/4 have the most say (because, let's say, 1/4 doesn't speak up at all, and the other isn't as assertive. Shades of grey, right?). If there is a gross imbalance of airtime spent in each session listening to a couple of the players, that's nowhere near as much as one that is more evenly distributed in terms of air time. I tire of listening to the same voices speak, sometimes you want to change the channel. A D&D player who is silent or rarely speaks is like an extra in a movie, not a headliner. Headline stars should have speaking parts, and in D&D, the players should all be the stars. I'm not discounting having a party face, like one usual RP headliner and a bunch of backup vocalists. But the backup vocalists need to contribute at least the harmony. Silence doesn't harmonize with anything, or rather, it harmonizes with everything. That is, silence is completely forgettable and discountable. And by extension, silent players are as well.
I'm not at all advocating people purge their own groups of players who contribute little, that's not what I'm suggesting at all. I'm just saying, I consider it to be a problem in D&D, and in that respect, player types that cause problems (such as grossly unequal airtime because they never speak up) are by definition problem players. There are many other types, including the spotlight hog who always wants to be the star of the show and monopolizes the game, seeing D&D as their personal theater stage with everyone else playing bit parts and them getting all the monologues and soliloquies. Those are also problem players, too. I don't have solutions to all these issues, or even any, aside from the obvious, which is to talk to them and ask them to share the spotlight and sometimes to grab it to get their fair share. For the good of the game. It's more fun for everyone when everyone's contributing. I feel sad for those who don't contribute, and that is a detriment to my own fun, so I'd rather just not be around such players. I'd also avoid players who are jerks, immature, spotlight hogs, but if you expect perfect players you will be left alone. I don't expect perfection, or even want it, I just want an equitable share of airtime and it's everyone's responsibility at the table to not only not hog it, but also to assert it and demand it for themselves. People who don't do that, ever, or very rarely, are a detriment to the game's potential.