In older editions of D&D, attempting to pick a lock would absolutely involve a random encounter roll. As would searching a room or X amount of time spent moving about a dangerous area like a dungeon.
Also, I've always looked at random encounter tables as a list of people/creatures that can be encountered in the given area. The roll doesn't conjure them into existence... they were always there... it just means that they're now right here, where the PCs are.
Yes. It's not at all required that the actual attempt succeeds when there was a failed roll... it's more that something unwanted or otherwise negative happens which creates a new situation. It's not about "the narrative" or "the story" moving forward. It's about the circumstances... the situation... moving forward.
So instead of opening the door and finding the cook, instead the thief is noticed by the neighbor across the street. Or the neighbor's dog begins to bark nearby, having smelled the intruders.
Any number of things. It really is just about not saying "nothing happens".
Because a player rolling for stats has nothing to do with my point?
You're saying that because you as GM don't want total control over a setting, you enjoy when a creature's presence is indicated by a die roll.
But that's what the lockpick attempt is doing. We're just having it do double-duty, so to speak. It's a die roll that tells you of the presence of the cook.
I'm surprised to hear a fiction first fan such as you ask that. I mean... it's obvious without looking at any rulebook, isn't it?