It is tied to HP being ambiguous, I think. Because HP can represent "luck", you can prevent losing luck by dodging on purpose, hence Dex being added to AC. Stronger armors replace this skill with a solid foundation, allowing you to bear the brunt of attacks despite not actually avoiding them. Of course, this means that a "Miss" in game terms could still be a narrative "hit" on your armor, while a "Hit" in game terms can be a Narrative miss. Doesn't bother me, personally, but I can see it bugging someone.
On your houserule, it would need to be a substantial number of temp HP. You never roll to hit again, but instead only roll damage. As a result, you need to figure out a general average number for how much HP a given AC normally defends. For example, assume a 15 AC protects a 1st level adventurer from 7 1d6(4) hits per short rest. So, the 15 AC would be replaced with the equation
[Level*average damage*Number of hits]=28 temp hp.
Of course, that is completely arbitrary on my part. My point is, you need to find out what equation you would use to decide how much temp HP. Thinking about it a bit harder, [Level+1*AC] seems like an alright area. A 15 AC would then provide 30 HP bonus for a first level character, but more for a progressively more skilled character. This would make abilities that give resistance, like Rage, much stronger, since it effectively doubles the power of armor vs. non-magical attacks. A bear totem barbarian, at 3rd level with an AC of 17 would basically have 68 extra health, doubled as long as they are not fighting psychic damage. That said, I actually like this thought, thanks for putting it in my head.
EDIT: To continue, you would still need to roll, but any result other than 20 or 1 can be treated as dealing damage, maybe even adding your roll to hit to the damage received. A 1 can be treated as a complete miss, while a 20 doubles/maximizes damage as normal for a given table. I am liking this thought more. I might come back and post another response when I have put more thought in.