A few ideas, expanding on the linked-to-Plane-of-Air theme from thewallshadow:
1) Perhaps the island is held up by 1001 Huge Air Elementals, bound to service for 1,000 years (or more, or less) by...
...a powerful caster
...an ancient compact or treaty that ended a war between the Material Plane and the Plane of Air
...a powerful elemental being, see later
The elementals can't be simply dismissed or slain, because...
2) The main bulk of the island is actually
in the Plane of Air, and is encapsulated in a planar bubble which makes it visible from the Prime Material plane: like a hole in a wall allowing you to see into a room. Depending on how hard you want to make this, it could (and probably should) be impossible to cross the bubble's boundary from the Prime Material, whether by flying to the base of the island or digging from its surface: it's just impervious to most forms of tampering. *[1]
3) Perhaps an Antimagic Field can temporarily suppress the boundary, allowing a character to cross... but when they do, they find themselves in a random point in the Plane of Air, quite possibly plummetting towards one of the few chunks of solid matter with local gravity.
4) The island and immediate environs are co-terminous with the Plane of Air instead of the Ethereal Plane. This has the side-effect that creatures who are ethereal before arrival can't approach the island any closer than whatever effect radius you want. Any spell or effect that would normally render a character ethereal when cast
on the island instead shunts them to the island's location on the Plane of Air, where they see something entirely different...
5) On the Plane of Air, the island is the domain of a powerful elemental entity such as the Sultan of the Djinn or somesuch. (Yes, that's right, it's a Djinn Palace

) He's aware of the island's use in the Prime Material and...
...doesn't care, or;
...is actively sponsoring its use in the Prime for whatever reason, or;
...hates the fact with a passion;
And so...
...he would be MOST upset if the party somehow unbound the elementals moving his palace around, or;
...he doesn't care what they do but is annoyed at having the party on his plane and gives them 48 hours to sort it out and get lost again, "
or else", or;
...he desperately wants to be free of the situation and gives the party a quest to find the MacGuffin that will permit him to finally pull the whole thing fully into the Plane of Air where it belongs
6) Should the party succeed in undoing the spell or agreement that keeps the elementals in place, the island falls/is banished to the Plane of Air: if the former, this has little or no effect on its existence in the Plane of Air, but brings it crashing to the ground on the Prime Material. This could be achieved by...
...getting rid of all the Prime Material entities on the island (by convincing them or tricking them to leave, or just stabbing them all in the face) so the ancient agreement binding the elementals to service ("for howsoever long shall a single mortal soul still abide upon its surface") will be fulfilled and the elementals just bugger off and leave the island to fall
...finding the Crystal Heart of Air. This is a MacGuffin - er, sorry, "minor artefact" - used in the weaving of the binding spell, which now sits in the Unlikely Location of Doom, guarded by Eldrtich Creatures of Not-Quite-Adequate Power. Destroy it or give it to the Ruler of the Djinn Palace to end the spell.
...Killing all the elementals: extremely freakin' unlikely, as a) if it were that easy, someone would have done it, b) only half of them are required to hold the island up, and a slain elemental is replaced 1d4 hours later and c) it's not remotely cool enough
...smashing the control rods at various locations on the island (or other plot-dictated Site of Special Arcane Interest), which can only be achieved by casting 20 levels of spells with the [Earth] descriptor at each one. The control rods probably regenerate (after an amount of time you choose that's inversely proportional to the level of the party: that's just how these things tend to work, it's like a law or something), making it a race against time to finish the job.
7) The party realise on a DC 1 Knowledge (Pretty-Much-Anything) check that 310,000,000 tons of rock falling from thousands of feet up in the sky might just possibly have a
tiny effect on the local ecology*[2] and decide that banishing it to the Plane of Air is a much better idea: at which point, see above.
Anyway, help yourself: I hope some of it's useful
***
*[1] A nice special effect, rather than just "an invisible wall of force", might be that as a creature approaches the encapsulated boundary, they are blown back by increasingly ferocious winds: extremely strong or massive characters might be able to get closer before being hurled away by the tempest, but still nowhere near crossing the actual planar interface. The howling winds are transdimensional, affecting even incorporeal creatures. This effect could give the party clues:
a) They can see birds attempting to fly to the base of the island: they get within a certain distance and then are suddenly thrown backwards as though in the teeth of a terrible gale, although the party can't sense any wind
b) If they view the island with
see invisible or similar spells or effects, they can detect small, vaguely bird-like forms landing on and taking off from the base of the island, which seem to appear out of nowhere as they arrive (they appear from "out of shot": a part of the Plane of Air outside the bubble) and disappear just as suddenly when they get a short distance from the island.
*[2] Such as "scything the area clean of every living thing in a 50-mile radius", for instance.