Paul Farquhar
Legend
I think of it as a speed thing... you could swing a sword slowly through a ghost and not affect it, but swinging quickly causes damage by disrupting the ether or something. Similarly I'd play that if a wall fell on a ghost, it would take damage.
It's psychic resonance. Even a non-magical weapon has a faint presence on the Astral and Ethereal planes due to the belief of the wielder.