Just a tiny bit of creativity can turn a dull, flat "nope, sorry, doesn't work, move on" into an amazing experience. This specific one hasn't happened (and isn't likely to; the time shenanigans probably won't come back), but something loosely analogous is likely in the future, and I can't wait to see how the vaguely-criminal-ish character (our party Bard) handles it.
And I assert that the "recognition" you speak of is much more fundamental than "oh yes, I literally already know your name, family history, rap sheet, and favorite dining establishment." That it can be built off of those intangible, ineffable, intuitive things that factor into nonverbal communication.
Conversely: Perhaps it is D&D--or, rather, a limited perspective of what D&D can be--that is at fault here. Perhaps we should exhaust as many avenues as we can reasonably try before finally admitting defeat, and that only with great reluctance. Perhaps, instead of viewing these backgrounds as a poorly-wrought, alien imposition that can and should be thrown off like a soiled jacket as soon as one is out of the rain, we should see them as a welcome pointer to new, interesting, useful things that we can leverage into excellent gameplay and/or narrative with just a few minutes and a dollop of creativity.
Perhaps we should be looking for a reason to say yes, rather than looking for a reason to say no.