I can only go on personal experience, but when I walk the Pennine way, I can tell you that every chance to take your pack off, you take.
Its not just the facts its heavy, its that it completely alters your balance. Even sitting down with a full pack on is pretty ill advised because it pulls you down in ways you really don't want to bend. Actually trying to dodge a sword blow or an arrow, I would think you would end up on the floor 90% of the time.
Based on personal experience, for a short time back in 1986 I was a member of the Society for Creative Anachronism, and trained to be a fighter in full armor swinging a rattan sword and wooden shield. The total kit weighed about 120 lbs, which is just short of my body weight at the time (I'm a little guy). Although it was exhausting wearing the outfit and in combat, I could only last a few minutes wearing all that while attacking and parrying blows, however, despite the encumbrance by weight, it was fairly evenly distributed across my entire body, so there was no off-balancing affect tending me towards being on my behind instead of standing on my own 2 feet. Admittedly, when exhausted after attempting combat for a few minutes in full armor, I wasn't able to hold the shield high enough to block head shots, as it became too heavy - but, even when exhausted, I wasn't physically compelled to be on the ground. At no time was this the case. Only a kill shot knocked you off your feet, not the weight of the armor. Note I wasn't very experienced and was only in the SCA two summers, so I was not as adept at doing this as others, so if encumbrance was an issue and would tend to keep you on the ground, it would be so for me, and it wasn't. I disagree with your statement, based on actual experience.
Wearing a heavy backpack with all the weight on your back (one area), instead of a full suit of armor, which is heavier, but more evenly distributed have no comparison in its affects regarding the wearer's balance. I was in the US Army back then, at the same time, so I am also experienced walking with a heavy back pack - that activity was indeed a balance issue, but not even close to the same thing as wearing full armor.