Certainly to some extent this is true. But in many ways we're still inevitably going to apply our Invisible Rulebooks.
If the stat block for the dragon says it weighs five tons, and the DM describes it as such, then when it lands on the roof of the house, the players' minds are generally going to process "thing that weighs five tons perched on roof top", with likely some impact on how they react to the scene. Perhaps I envision the roof as groaning and creaking under the weight. Perhaps I upgrade my previous imagination of just how big and sturdy the house is. Perhaps I ask the DM "Does the rooftop creak or break under the weight", with an eye toward suspecting it's an illusion if he says "No, not that you can see."
I definitely agree that the degree to which we can apply real-world physics and similar life experience and knowledge is reduced by the fantastic and the magical, but I think it's something we tend to do anyway, and I think a design tends to be easier to "grok" if it tries to be compatible with our Invisible Rulebooks where possible. I think this is generally a rule of thumb for fantasy and speculative fiction too, isn't it? That the world works the way our world does, unless and except where specified otherwise?