Just have Sauron put them on the Terrorist Watch List.
The Eye of Sauron as surveillance technology would fit JRRT's theme of industrial technology as a corrupting influence on the idyllic English countryside.
Mordor - and Isengard - should be industrial; the final journey should cross a toxic wasteland. Maybe a Siberian facility which also produced nerve gas, and poisoned the landscape for miles downstream? Or Chernobyl?
Who's Saruman in this mess? Going with BRG's idea, maybe it's like the HYDRA take-over of SHIELD, so you have the moral complications of opposing former allies while hoping for their redemption?