Assuming magic is a "baked in" aspect of the universe in the same way that spacetime and whatnot are, (perhaps, eg, a "fabric" or "the Weave") then maybe Force damage arises from destabilization of that magical medium? Maybe a sort of entropy effect in the magic medium?
So as a low-power manifestation, magic missile just sort of jostles the magical medium in a target before it snaps all back into place, leaving little more than a painful welt on the body or concussion in the brain.
Disintegrate, at its most effective, violently disrupts that magical medium, leaving the target completely dissociated; whereas in the lower-damage case leaves a target generally pulped, singed and/or discombobulated.
On the other hand, force objects like wall of force and so forth, arise from a controlled "crystallization" of that magical medium-- the opposite of an entropic effect. The spells are all about shaping it and cranking its "hardness" up or down. And when hit with a force damage spell, the result more often than not is just a cancellation of the two effects.
Anyway, my 2cp. I never really thought about it. Until now, I've envisioned it as just bludgeoning-but-not-really-bludgeoning or something.