The whole point of abstraction is to not sweat the small stuff. But as I've asked many times before, what type of system would you want? I don't see a better way of handling perception and I think ability modifier plus proficiency is as good a model as any especially for something like perception. We humans do not pay attention to everything going on around us, we just don't.
Beyond that I'm at a loss for what you're even trying to say. Are there systems with more detail? Of course. I just don't see how a chart or lookup table is any better, it just changes who provides the reason for failure to the author that cannot know the current situation. There is also no way a detailed system can cover everything a character might attempt. If there's a better system, give examples of the actual detail of the system and how it works.