I think this thread has led me to the conclusion that the engineering metaphor ('mechanics') is fundamentally misleading when trying to understand the ways that RPG systems change over time. I don't think analogies with self-serious high arts such as literature are any better.
I think a better core metaphor would be fashion. In year X, why is everyone wearing skinny jeans? Because someone designed a pair that looked really good, and someone influential wore them in a film, and then everyone wanted some of that action. In year X+10, why is everyone wearing baggy jeans? For the same reasons, plus the skinny jean look had got
so tired, and the ones in the shops just weren't ever as good as that classic pair from the film anyway.
Like
hawkeyefan says sometimes newer
is better, but that's true in fashion too. Changes in materials mean that modern sportswear does its intended job far better than stuff from the 70's; and, if you want, you can style it in ways that call back 70's designs too.
The fashion metaphor much more easily accepts subcultures than the engineering or high art ones. Just as punk, hip hop, and reggae could all be called modern in 1977 (don't ask me about nowadays, I'm old); so too can Shadowdark, Brindlewood Bay, and Daggerheart all be called modern now. I think this avoids the subconscious need to make 'modern' into something internally consistent, which it isn't really.
The word 'modern' is instead an umbrella term for stuff a whole bunch of disparate groups, playing in often very different ways, think is cool right now. Playbooks are modern, but so are the OSR's overloaded encounter dice. Both call back old designs (class and encounter rolls respectively) without simply restating them, and both appeared over roughly the same time period, but you won't generally see them in the same game because they're from different subcultures.