Yeah, you've got it.
In practice, I think you'd still do a "second" roll vs. damage avoidance, which is a straight roll d20 roll, preferably with no modifiers. This way, you have the ability to avoid damage 25%, 50%, or 75% of the time. It's easy to remember (not hard to track), there's not...