FormerlyHemlock
Hero
Yes remaining! I keep seeing "per x feet traveled", but remaining makes more sense.
This is also a much saner way to run air elemental Whirlwind, giants that hurl PCs around using Storm King's Thunder rules, and other monsters that shove for damage. I don't know about the specific numbers (d4 per 5' remaining is plausible but I haven't thought deeply about it) but the basic idea is good: unused movement converts to damage.
Just watch out for the interaction with Agonizing Repelling Blast!