The D&D rules do not attempt to answer this question. The goal of the rules is to make it clear in 95% of cases what a given spell can or cannot do (with the remaining 5% being left to DM adjudication). The rules are not trying to explain how the spell does what it does, nor to formulate a consistent theoretical basis for magic*.
IMO, this is very much the correct approach. <soapbox> There is a widespread idea that a magic system where the principles of magic are laid out in detail is more predictable than one where no such principles are given. What I've noticed is that this is not, in practice, the case. Just like real-world science, there is a vast gap between theoretical first principles and actual practical application, and you can stuff all kinds of "magic technobabble" into that gap to justify whatever ass pull you have in mind. Trying to evaluate the technobabble based on first principles is like trying to use particle physics to evaluate your mechanic's explanation for why the car won't start. </soapbox>
For a magic system to be predictable (which an RPG with spellcasters very much needs), it must specify not principles but outcomes. The rules don't need to say how the magic works. They do need to make it clear that spell X can do Y and cannot do Z.
And in saying that spell X can do Y but not Z there IMO also needs to be a
why in there somewhere. Put another way, if the rules don't need to say how the magic works then I-as-DM am left to make it up for myself; and having at least a vague idea of what the designers had in mind would make that process much easier (and if the designers didn't have anything in mind that's an unrecoverable fail on their part).
It's trivially easy to lay down a few basic principles:
--- if a spell causes anything physical - a bead, an arrow, etc. - to go from caster to target(1) then any physical obstacle will block it
--- if a spell does not cause anything physical to travel from caster to target(1) but requires a visible target, then transparent physical obstacles do not hinder that spell, nor do they hinder visible spell effects e.g. rays, dancing lights, etc. from functioning as intended
--- if a spell neither requires a visible target nor causes anything physical or visible to travel from caster to target then it can be cast anywhere within the caster's range without regard for obstacles of any kind(2)
--- a roll for aiming is always required when caster cannot see the target point, person(s), place, or area.
1 - or from somewhere else to target e.g.
Call Lightning causes lightning to travel from the sky to the target
2 - excepting obstacles that block all magic e.g. a lead-lined wall; and this is specifically intended to allow blind-casting into previously-unseen areas
*They put a little bit of theory into 5E with the sidebar about the Weave. No previous edition felt the need to bother with this; the Weave was a Forgotten Realms notion that didn't apply in other settings. And even in 5E, that sidebar ties into nothing else in the rules and can be dispensed with at will.
Yes; and though I've never 100% bought into Weave theory myself I appreciate it as at least being a halfway-thought-out attempt at explaining how and why magic works the way it does.