If you're playing/running Isle of Dread as a one-off adventure then sure, what you say is absolutely true.
But I'm assuming in all cases here that these adventures are embedded in an ongoing campaign...which in this case means the PCs were in X for a reason (even if only to get on the ship to sail to Y) and are...were...intent on going to Y for a reason. That they were in X means I've had to narrate at least a bit of description about the place; and whatever Y is would have to have had some sort of narration or description or clues because otherwise why would they be going there?
We're also told that it's far from the nearest settlement and away from travelled routes, strongly implying that the PCs will not be easily able to go back to town and resupply and-or recruit new characters to replace their dead. Because of this, and because low-level D&D play tends toward resource management, a DM is going to want to know how many days it takes to travel from town to the adventure site so as to monitor the PCs' food supply...which makes positioning both the dungeon and the "nearest settlement" on a map a rather useful thing to do. Noting any intervening terrain features that may help or hinder travel would also be useful. It's also trivially easy to do these things; then later as the campaign goes on they can be expanded upon until soon enough you've the genesis of a game world.
Me, I just prefer to move that work from within the campaign to before it starts. That way I can think through the in-play implications of what I've designed* and tweak it to suit.
* - a process I manage to mess up at every opportunity, but hey - live and learn...
Lanefan