If the PCs perform a statistical analysis of the events dictated by the game mechanics, they're going to see the game mechanics, and it's going to be odd. Fortunately, we're the ones in charge, so we can ensure the PCs never perform such an analysis.
If the PCs set up a double-blind study using hindering terrain and an illusion of hindering terrain, they'll find that forced movement is thwarted by a drop-to-prone 55% of the time for the hindering terrain, and 0% of the time for the illusion. "Isn't that just bizarre?"
But if they don't run the analysis, all they'll know is that while you can sometimes throw yourself to the ground to avoid falling in the pit, it doesn't always work. And it happens that the three times in a particular character's 1st-30th career that he was pushed into a pit that turned out to be an illusion, it didn't work.