Or known...I'll go back to the history check. I outlined a real cost to not remembering the history of the McGuffin. I guess I'm not sure why you're being so dismissive because of the nature of the cost. No, it's not immediate but it is a cost. It's a cost of time if there's a race against the clock, it's a potential combat encounter or monetary penalty depending on Jimmy's mood.
Let's say I'm trying to remember how to get to the local tavern. I try to remember the shortest route and forget that I can just go left at the next corner and get there in 5 minutes. Instead I take the long route and it takes 25 minutes. I miss happy hour in addition to spending an additional 20 minutes walking I didn't need to. If my memory is bad enough I might end up at a healthy juice bar (possibly a fate worse than death) instead of the tavern.
So for sake of argument if I want there to be an explicit penalty for a failed knowledge check then that penalty does not have to be immediate to be significant.
Failure to pick a good route can mean getting their a little later but also maybe picking up unwanted attention, getting into a section of town that is problematic.
I mean, how many tropes of "someone saw the wrongbthing" or someone thinks they did do we have out there?