The issue comes when the action is so specialized or lore is so secretive that the nonspecialist could not have perform the action nor happened across the info.
Therefore the fiat of the DM to determine the impossible becomes more important.
It's not a bad thing but it's a new thing to think about.
I agree, but I still want to push back on the "impossible" idea, by saying that I've been a specialist in a few fields, and there is very little knowledge that is actually impossible for a (generally) non-proficient person to randomly know something about and very few tasks that are actually impossible for a non-proficient person to, with a lot of luck, accomplish, if they actually attempted them.
What there is, is a whole lot of things that are much more improbable than a 5% chance that a non-specialist would succeed at. So the decision DMs have to make is whether they are okay with the level of heightened reality that enables unlikely characters to succeed at unlikely things (sometimes incredibly unlikely things) 5% of the time, or whether they need more verisimilitude on this front in their magical fantasy game, in which case they will need to be more careful about allowing people to roll for things.
Basically what you have to do is, before allowing a roll by a character for whom success feels like it should be impossible, think about whether success at the particular thing by that particular character would be inconsistent with the tone of your game.
Personally the only things that come to mind where I am uncomfortable with the "wrong character" succeeding at in my games are checks that I am only allowing to another character because of their backstory, and attempts to perform with a musical instrument by someone who has never played any related musical instrument (being a natural is okay, but you aren't going to figure out the lute on the first strum).
I think improbable checks are generally better gated behind character background than proficiency. After all the 5e skill system does not generally make much logical sense (if I have a background in religion my religion skills grow with my general proficiency from experience in non-religious studies related adventures?) and it seems like anything made possible to succeed at through a general proficiency in one of 5e's very general skills should not be completely impossible for someone who is not proficient. But if the reason you allow someone to roll is not because they have the Arcana proficiency, but because their background is that they acquired that proficiency at a mage school and it seems like they are rolling for a piece of arcane knowledge that, in your setting, virtually nobody outside of magic schools would ever know, then it makes a lot more sense to gate other characters off from that roll.
