I'm gonna say pirates. They have ships and cannons, and I'm not sure that ninjas can swim.
And if they DO swim, their tight black pajamas will make them look like seals, resulting in mistaken-identity shark attacks as they try to swim up to the pirates ships.
It all depends on how many ninjas there are... Remember the Inverse Ninja Law. "The total Real Ultimate Power of any given group of fighting ninjas is inversely proportional to the number of ninjas in that group."
So, if there are only one or two ninjas going against an entire ship of pirates, the poor pirates don't really stand a chance.
But, once the ninjas start attacking groups of a dozen or more, they suddenly won't stand a chance against cannons and pistols.