It is task dependent. Failing while climbing a cliff, especially if it's a cumulative skill challenge, can be dangerous. Failing to persuade someone might make it impossible for them to trust you again in that moment. Heck, even when failing to pick a lock the first time, I have seen DMs say, "You tried, but it just eludes you. It could be the lighting, the rust on the lock, or something is jammed in it. But you can't pick it." And therefore, negate any future attempt in that specific setting. I don't see any problem with this.
I personally like two standards when dealing with skill checks: Some skill checks require you to have proficiency before attempting, and consequences, however minor, should be the result of a failure. The reason for the latter is, there are a host of spells, class abilities, and species-oriented abilities that help or negate skill checks. So, when the exploration pillar of the game does appear, it should have a clear purpose, a natural consequence, and an alternative if the skill check is needed to continue the plot.
In the end, the skills are so broad, that their specificity is what matters - a sometimes yes, sometimes no situation.