I think the difference is mostly one of philosophy.
To my mind in a rules heavy game the rules themselves seek to channel the gameworld, such that engaging with them is the same as engaging with the fiction directly. I'm thinking here of Rolemaster/MERP, where you select how to split your combat bonus between attack and defence, and things like that. There will be a bunch of detail about positioning and modifiers because we want to see and incorporate the various factors that could have an influence on the outcome.
In a rules light game the users are expected to provide that kind of input themselves. Not just expected to, they are assumed to want to provide that input. The job of the rules is to channel the group's idea of the fiction, to allow those inputs to be fed into the machine and an outcome determined. So the users may decide what the stakes are, what the difficulty is, and then consult the game engine to determine the result. That result can be just as determinative as in a rules heavy game (often moreso).
In both cases the rules are not a 'necessary evil' but a key and valued part of play. Games where the rules are just a necessary evil, to be disregarded, fudged, or skipped over as quickly as possible, have no value.