Yeah, it's almost like we need a "z axis" to determine how difficult a monster could be, given certain external factors. For example, a cave bear is a cave bear. You can generally look at that creature and know what to expect from the stat block, and CR should be fairly accurate given nearly any sort of "typical" circumstance (finding one asleep or at the bottom of a pit notwithstanding). On the other hand, the gelatinous cube is either trivial or deadly-up-to-TPK, depending on the environment. It's almost like you could use a color-coded system on top of CR; something like green for "fairly straightforward," yellow for "can be more dangerous given an ideal setup or if your party lacks a key ability," and red for "given a relatively common setup or without a specific type of counter this monster can be far more deadly than its CR might indicate."
Of course, the more you try to codify this stuff, the more confusing it gets, the more words you have to dedicate to explaining it, and the cost of everything gets higher - possibly without much gain, since most experienced DMs would likely intuit this stuff anyway.
Such a system would almost seem unwieldy. There are just so many combinations of stuff in stack blocks versus situations that could occur. The CR system seems to be based on "everyone take your turns and do your most deadly thing. Anything outside the scope of that falls into encounter building where the CR is just an XP value, but the extra stuff can modify the encounter in fun and interesting ways.