As someone who is relativly new to D&D and TTRPG in general I did not know that about OSR games thank you for that information.
as for the micro math that is a concern and would in general slow the game down which combat is something that can take a good chunk of time as is.
Feel free to experiment as you're feeling out the style of game & house rules you like. "Players, I'm trying this thing out, don't know how it will go, may need to course correct, is that ok with you?" If you have enough gaming time to have a point of comparison for how long baseline / rules-as-written combat plays, give it a shot.
It's hard to ever say that a house rule is "good" or "bad", because the same house rule might play differently between two different groups due to play styles, GM rulings, and party composition.
I have though about HP or durability of armor and perhaps that is a bit much it was more of balance idea and something to sink gold into since in 5e gold is kinda useless but I have kinda fix that problem by giving out less and making mundane items more valuable and adding in additional options for mundane items.
To minimize additional tracking of points, you could just have a "sundered/broken" state where the armor ceases to function. Placing an X on your sheet or a mental flag "armor is broken" is probably easier than tracking another pool of points.
As someone did point out there is a feat in 5e "Heavy armor master" that does a flat 3 damage reduction which I think is not worth a feat but I could slap it onto medium and heavy armor? It is a small amount but over time it does end up helping a lot also would be easy enough to calculate?
The idea of rolling a d4 is because players and myself just like to roll dice lol but perhaps a flat amount is faster ?
5e's is a "rules ecology" or "rules dominos" in that one rule here can trickle over and effect another rule over there, and when designing / house ruling you often need to keep other rules in mind because they interface with the rule you're working on. Very few rules in 5e are totally isolated.
On paper, you're right, Heavy Armor Master is kinda underwhelming. However, remember what I said about 5e being full of characters/monsters making multiple attacks each round? Multiattack quickly becomes a standard ability monsters have. And against groups of enemies where the tank PC is at the frontline receiving more focused fire than other PCs? That's more attacks. And that means more chances for that 3 damage reduction to kick in
on each attack. So it's not just potentially sparing you 3 damage per round, but maybe more like 9 damage per round, 12, or even more.
So, applying that to all Medium/Heavy armor is a significant boost (depending if you're taking away anything, like replacing AC with a soak system).
As far as making it a die roll instead of flat 3... again this is a good time to look at other rules that might impact gameplay. One that comes to mind is the Battlemaster's parry maneuver, which involves a reaction die roll; I could imagine a scenario where the Battlemaster rolls to parry (reducing damage), then they roll d4 for the armor damage reduction. Now imagine that Battlemaster facing two hobgoblins, ok that's a die roll to parry, and 2d4 rolls to soak. NOW imagine that Battlemaster facing two githyanki warriors with multiattack, ok, that's a die roll to parry, and 4d4 rolls to soak... So you need to figure out how much handling time that's adding and whether worth it for you.
Maybe for your group that's happy to roll, that's fun! Go for it!