Let's suppose a scenario. The party steps through a portal and is transported 5,000 years into the past. What features would actually work? There's obviously no contacts, criminal or otherwise. There may or may not be libraries for that sage to investigate. Even if you're Prince Grand High Poobah, it's of a country that won't exist for another 4,000 years so it's meaningless. Assuming the locals even recognize nobility as a thing.
As I said above: like recognizes like. This is literally written into the description of the ability. You look like you fit in, even if you don't actually fit in, because you have that
je ne sais quoi aristocratic air. (Which, of course, can bite you in the butt too--you can't turn a sow's ear into a silk purse, but a silk purse stands out
pretty hard in the pigsty.) Certainly, things could be much more complicated, at the very least a language barrier is likely.
But guess what?
I've literally done something like this. And it worked! It was actually super cool, particularly because it put a frequently quiet player in an interesting role. So the fact that you think this is an utterly ridiculous, unbelievable scenario when I've not only done it but gotten a shy and casual player to open up some more and genuinely feel deeply invested as a result tells me
maybe you shouldn't dismiss this as some utterly ridiculous, impossible example.
(Formally speaking, the character has class features for diplomatic stuff rather than a 5e-type Background, and it was a bit of time travel
very roughly 3000 years into the past rather than 5000, but I hope you'd agree that those are rather less important than the event itself.)
There is no logical reason for someone to have a criminal contact, at least not right away. That pirate that can get away with minor crimes because of their reputation is just as unknown as everyone else. The archeologist might be able to tell people approximately when we are because of their training. The noble might have a better idea of what the social hierarchy is and even have advantage on interactions with the ruling class. But if they want to get an audience with the ruler of the land, which may not even have the concept of nobility in terms of inherited titles, claiming a noble lineage of somewhere that doesn't exist is not going to buy them anything.
Criminal
contact, no. But just as above, like recognizes like, and you can't tell me that some cunning bastard wouldn't see the potential there and try to
do something with it. Not only is that a
sweet narrative moment, it's actually quite realistic--because when you're a prince-of-thieves kind of person, you learn how to size people up right quick, and more importantly, how to see potential where others see none at all. Seeking out and investing into a professional relationship--"networking," one might say--is a big deal, whatever stratum of society one calls home.
Just a tiny bit of creativity can turn a dull, flat "nope, sorry, doesn't work, move on" into an amazing experience. This specific one hasn't happened (and isn't likely to; the time shenanigans probably won't come back), but something loosely analogous is likely in the future, and I can't
wait to see how the vaguely-criminal-ish character (our party Bard) handles it.
It would be jarring to me as a player and be completely illogical world building if all of our background features still worked as written. Many of the background features are based on recognition, contacts and understanding how things work where people are from.
And I assert that the "recognition" you speak of is much more fundamental than "oh yes, I literally already know your name, family history, rap sheet, and favorite dining establishment." That it can be built off of those intangible, ineffable, intuitive things that factor into nonverbal communication.
There are times when background features as written won't work. Doesn't mean the feature is totally worthless, it may or may not be. But these features are not built in to the framework of D&D, IMHO they're tacked on. Many are poorly worded or thought out and many only apply where the person would logically be recognized. I'll take a ruling from a DM that makes sense for the scenario we find ourselves in over strict literal reading of the rules for background features any day of the week.
Conversely: Perhaps it is D&D--or, rather, a limited perspective of what D&D can be--that is at fault here. Perhaps we should exhaust as many avenues as we can reasonably try before finally admitting defeat, and that only with great reluctance. Perhaps, instead of viewing these backgrounds as a poorly-wrought, alien imposition that can and should be thrown off like a soiled jacket as soon as one is out of the rain, we should see them as a welcome pointer to new, interesting,
useful things that we can leverage into excellent gameplay and/or narrative with just a few minutes and a dollop of creativity.
Perhaps we should be looking for a reason to say yes, rather than looking for a reason to say no.