(Actually, the ship can be repaired; it isn´t too strange to be stranded on an island with a badly damaged ship, and having to repair it, or even building one from scratch. However, that takes months, or even years if the crew has to buid a temporary village and grow crops in the meantime. If there´s hurry, and they have two or more ships, sometimes a ship is cannibalized to make the reparations faster, but that´s an option you don´t have. Anyway, giving three days seem to be more a symbolic thing than a serious offer, as they know your current state and know perfectly that it´ll take a long time to repair the ship. It´s no different that saying "we´ll kill you".
About their numbers, judging by the town´s size, if they are serious about making war with the jolly dolphin´s crew, things are going dark for you. Maybe the party could put a good fight, but you´re afraid the average sailor, while experienced, are no match for them, and you are badly outnumbered.)