I think it's simple a result of relatively lazy thinking early in D&D's history which has been continued as a trope, rather than any conscious design decision.
Early on, spells were pretty scarce and seen to be important. And obviously casters can't "silver" a spell (where others can swap to a silvered weapon or the like). And on top of that, in fiction, casters tended to be able to harm various creatures of the night. So I think it was essentially seen as "only fair" that spells hit these creatures normally, even if it didn't entirely make sense. Note that the reverse was true somewhat as well. There are some creatures which are immune to certain spells or damage types largely associated with spells that clearly haven't really been thought through either.
As time has gone on, spells have grown more common and in some ways more powerful, certainly more usable, so it's more of a question, and yes, has made it increasingly pointless to use silvered weapons or the like. I do think changing to regeneration rather than resistance or immunity makes sense in part because of that. Regeneration is still an annoyance to casters, where if something isn't resistant or immune to that damage type, or magic damage in general, they didn't have to be concerned.