Another thing: It is actually not a decanter, but a fountain of endless water. It don't costs more to create than a decanter and a fountain separately.
If people try to steal it, more power to them, but they'll have to bother with the transport a one-ton stone monument.
If people think there's a decanter they can steal, and break through the fountain, then they'll destroy the magic item, and thus, the magic.
Similarly, it could be actually an enchantment (yeah, I know, a conjuration-creation) keyed to the place. If they break everything, they'll find a freaky geyser that starts out of thin air, 1 foot above the heap of rubble they've made.
A last option would be to have the stopper lost. It seems nothing, but try putting a mundane stopper on a geyser -- it'll pop out far away fast. The hassle of walking around with a unstopperable geyser is probably more than it's worth. Not to say how easy it would be to track the thief, leaving a wet trail behind them (they could fled through the river, though). And the incredible stealth they would have...
What if they find a convincing way to stopper the decanter ? Well, it's not the magic stopper, so it don't stop the water generation... And water don't like to be compressed. One of the thing, stopper or decanter, is going to burst and be ruined.