Architecture suggests that four pillars are perfectly stable.
Well, that's sort of the thing, innit?
With three, everything is vital. You can't lose any of them without the structure becoming unstable. With four, you could have at least one that's a rickety wreck because it's not essential. Restricting yourself to three means you have to make sure they're all worthwhile, or else it's quite noticeable that they aren't.
"Downtime" is part of the social pillar, and one my players show no interest in engaging in, and therefore not a required part of the game. Same with kingdom management stuff. These things are just optional extras for those who want them.
Yeah, I definitely get the feeling that there's a lot of "I really REALLY like this subsystem, so it should be promoted to being one of the things
everyone should expect to care about."
For my own part, if I absolutely, positively
had to add a fourth pillar, it would be
Ethos. Note, not "Ethics," nor "Philosophy." Because that actually rises, to some extent, to the level of a key part of the game that everyone is expected to care about at least to SOME extent. Alignment goes in there. But so do things like values, personality, and "thematics" (e.g. choosing to play an "ice mage" so you only take spells which fit with that theme). 5e Backgrounds, 4e BGs/Themes/alignment, and prior games' alignment and other systems, all feed into this pillar. It has real, meaningful mechanical impact, and for all but the most stridently murderhobo near-zero-RP games, SOME part of Ethos matters to play.
In general, I don't think of this as actually being a pillar, but rather being something that rests
atop the pillars, a further elaboration of gameplay "above" the pillar part, since players choose whether and how much to dip their toes into it. But if I absolutely had
no other choice but to pick a "new" pillar to "add" to the game, that would be it. Because it's effectively already there. It just doesn't get called out because it's considered more a rider of play rather than a fundamental goal of play.