It strikes me (no pun intended) that the problem here is how folks are thinking about what attacks and rolls represent. You aren’t making an attack roll because you’re swinging your sword — you can do that because you have a sword and the physical ability to swing it — you’re making a roll because someone or something is attempting to stop you (actively or passively) from doing damage with that swing and thus making it important for the sake of the game’s math. Those of us who grew up playing pre-d20 will recall the one-minute rounds that presumed characters were swinging and parrying back and forth but only getting in the number of opportunities for key strikes that their Attacks Per Rounds affording them in game terms; the six-second round doesn’t enunciate this fiction the same way (although it really should, just on a shorter timescale).
Magic is the same way, in that a contested roll (contested either against another character or against a DC) not about you casting it but rather about when it interacts with someone/something seeking to defend against its effects. That could be represented as either an attack roll by the caster or a saving throw by the defender (it’s implicitly the same, just a matter of which defense is in play — one could devise a system where AC saving throws by defenders replace physical attack rolls in some manner, as the issue is a character reacting passively to the threat against them). Of course the other issue with spells, as noted by some commenters above, is their scarcity — certain attacks (spells, special abilities, even ranged weapons vis a vis ammunition) are limited versus swinging a sword all day long; different editions have different rules about stopping the casting of a spell (and whether it’s lost), but that scarcity of spells means that the “cost” of casting isn’t reliant on a check (even if whether the spell’s attack connects might be).
Something I’ve noticed over the years is the ebb and flow of how much to rely on the dice — I always look back with a pained grimace at how mechanistically 3.5 developed in terms of creating rulesets for every possible situation (hence why I could never get into Pathfinder or 4e, both of which rulesets continued along that overly-programmed route) — and whether DMs ask players to roll for everything versus limiting their use to only those situations where chance truly alters the outcome of the game (I’ve compared it to Two-Face in old Silver Age Batman comics flipping his coin for literally every decision). Limiting where checks are made allows for player decision (not just DM fiat) to fill in the gaps, and such rules on magic — and the freedom to cast it — are a part of that.