If a pit trap opens up beneath your character's feet, and someone needs to roll a die to see if they fall in. Should that be the player rolling a saving throw, or the DM rolling an attack against Reflex/Dexterity? In that case, in-world the saving throw makes the most sense (to my mind) - the trap isn't acting with intent.
On the other hand, if your character casts charm person on someone, maybe you should be rolling against their Will/Wisdom defence.
Things can get gummed up, in terms of translating what is happening in the game into mechanics, with things like fireball or a dragon's breath weapon, because attack rolls vs a (mostly) Dexterity-based defence is already a thing (attacks vs AC). But is that really the best mechanical representation of what is going on? There's a certain expectation of specificity to an attack roll, to my mind, that most area effects lack. When you launch a fireball into a room, you're not targeting anyone in particular - you're filling the room with magical flames, and it's on everyone in it to take the brunt of it or somehow avoid doing so. That again speaks to a saving throw.
So I think the most accurate way of representing such effects in the game is to have both non-AC defences and saving throws. But that certainly does not, to my mind, seem to fid the design aesthetic of 5e, and would probably be too fiddly and annoying in other games. Best to use one or the other - personally, I think saving throws works better overall for that purpose than attacks vs non-AC defences.