The entire spirit of the argument against these backgrounds is steeped in DM antagonism. DM antagonism that D&D has sowed and nurtured for 50 years.
That's the point of the exercise: the backgrounds give players narrative agency and that's not allowed.
If I had a nickel for every time I've heard a DM say "I don't think you should be able to do that" or "That doesn't make sense to me", I'd have a lot of nickels.
Not counting the ones I already have, because I've been here, and I think even well-intentioned GM's struggle with this sort of thing. Early on in my DM career. my peers instructed me in the school of "never give players an even break".
My very first campaign was a disaster because my players walked all over me. So I took this to heart and tried to be very strict about the letter of the rules and not let players get away with any shenanigans.
Now, what changed was, this wasn't really much fun. Sure, I'd learned the rules backwards and forwards (having read my rulebooks over and over again) so I wasn't going to get the wool pulled over my eyes, but this wasn't very much fun. Players stopped trying ingenious things because I'd always shoot them down by pointing out all the ways they wouldn't work.
Once, annoyed by instance #391 of the players cakewalking through my bad guys, I complained to another DM, and he gave me the best advice ever. "Why is it a problem when the players win?"
That thought made me stop and rethink my whole approach. Because the problem wasn't the players winning- I wanted them to win, or at least, if they fail, do so in a manner that keeps the game going. So what was the actual problem?
"Because they aren't winning the
right way." I said to myself, then realized how ridiculous that sounded. So only the solutions I allowed could work? No, that can't be right.
If the bad guys are taken out easily, there's always more bad guys. Always. There may be consequences for an easy victory if you step back and take a long view at the campaign.
If the players come up with a creative idea, what's the real harm in it? That they'll use this "exploit" over and over again? Who's to say they were the first person to ever think of it? Maybe it's come up enough times before that people have thought of countermeasures?
It's not easy to be a fair GM- a bad ruling could set a precedent you'll regret. But good players aren't going to hold your feet to the fire- you can admit you make a mistake. Or say "hey guys, that exploit is great, but I can't think of a way around it, can we maybe agree not to use it? Otherwise I might have to have my NPC's use it against you, and that might not be so fun."
If a player wants to say to me "hey, my character is a street urchin, would I be able to find a place the pickpockets and street thieves hang out?", "Oh I don't know, you've never been to this town before" may be a logical response, but it's not a very fun one (to me). My narrative isn't more important than my players, and if somehow this spoils a major plot point, maybe I should think about my plot points more carefully.
I mean low level magic spells do tons of damage to plots as it is, forcing you take them into account, so it's not like it's a unique situation. Granted, spells have costs (usually) and background features don't (and maybe they should be fueled by some meta-currency if they manage to be disruptive), but it's just in my opinion another tool a player can use to interact with the game world, no different than a 7th level Battlemaster looking at some guy for a few minutes and saying "yeah, uh, that 'ordinary town guard' you guys are thinking about sneaking past? He's a better warrior than I am!".
If I wouldn't interfere with one ability, I don't see any reason to interfere with the other.*
*That someone might have issues with the Battlemaster ability is a whole other kettle of fish.
