He explicitly labels it a guideline. Given the lack of other guidelines, it is THE guideline. Note how Elves & Halflings 90' away from a party and not in metal armor move so silently so as to have a better than normal surprise chance(pages 16-17 of PHB)? Why 90'? Where did that number just get used?
It makes perfectly good sense. There's a 1 in 3 chance to achieve surprise, normally. If you simply move normally and wear metal armor, you can be heard from 90', and you have a 1 in 3 chance to surprise. If you move relatively quietly, the distance is shrunk to 30', which is closer than the normal encounter distance I note, meaning a relatively quiet party will generally have a chance to avoid an encounter, as their opponents will not have heard them yet, though being surprised could negate any advantage of that. Elves and Halflings (and rangers even more so) operating well in advance of a group, regardless of if it is relatively quiet or not, are more difficult to surprise and gain surprise more easily, presumably because they are 'extra sneaky'. This just means that they can often sneak right up to within 30' of an enemy (the distance that an encounter happens at for the surprised party) or choose to avoid more reliably.
I still don't see anything that establishes a rule that metal armor means you are automatically able to be detected 90' away, and phb/dmg doesn't refer to metal armor specifically when discussing this, just to non-elf/Halfling characters. It isn't even clear what the rules are for a party of ALL elves, though we can assume they simply gain the benefit as that would be the logical conclusion.
No, Hide in Shadows(page 28 PHB) explicitly states that unless there's a heat-generating light source nearby the creature with Infravision or the creature attempting to hide(which prevents hiding...), it doesn't work against Infravision.
Mmmmm, now you want to delve into the ugly of infravision! Really? Are you sure? I refer you to the DMG section on vision for your interested reading. There you will get a LOT more nuanced treatment of infravision. You will learn that it isn't just 'normal vision but in the dark'. You'll also learn that there are FLAVORS of infravision, even though the MM doesn't discuss this point! 90' infravision is MUCH different from 60' infravision (which is what all PCs have, though I'm not sure about Drow). In any case, either type will definitely make Hide in Shadows more difficult, I agree. Its still up to the DM exactly what happens, though certainly PHB p28 suggests that thieves probably should just not bother to pursue their craft in dungeons. Still, my original point, that parties have plenty of scouting and stealth options, stands. A ranger is perfectly capable of moving up to within 30' of many monsters, and can certainly at least move to a point which is out of their line of sight and attempt to listen, smell, take a peek around the corner, etc.
Truthfully, if you want a totally sensible system for all of this, you should write up your own, because 1e's just isn't coherent. For instance there's actually no real discussion of how likely it is for say a totally silent motionless skeleton to surprise a party. How would they even know it exists? Yet surprise is still checked in the standard way and so is encounter distance!