Now, if we're going to include "bunker" in our definition of castle, things get a bit difficult to talk about. After all, is a cave in the ground fortified by whatevers, really a "castle" anymore?
The distinction of a 'castle' seems to be that it is a place of residence. Beyond that, it gets harder to define.
In my mind, when someone mentions castle, I'm thinking high walls, towers, bailleys that sort of thing. The Citadel in Halifax, Nova Scotia is a modern adaptation of a castle (and something that IMO, would likely happen to fantasy castles) but I'm not sure I'd call it an actual castle as such.
Other than its star shape, there is no particular feature of Fort George that is unknown in medieval architecture (extending into the high middle ages, that includes the cannon loops). What's different is the degree to which you find these features combined and used. And indeed since it dates to the first half of the 16th century, the star fort is at least as medieval as fully articulated platemail or three-masted ship rigged sailing vessels.
However, the star fort is roughly irrelevant without cannon and firearms. Everything about the star fort is designed around the cannon. A fantasy castle would have no need to spread out to create defence in depth, because it doesn't have the kilometers long range of the cannon to allow for active defence in depth. So a fantasy castle might well be star shaped to maximize fields of fire, but its going to be a relative small space if its going to dual with trebuchet and fireballs and defend itself with arrows because the interlocking fields of fire will be much shorter. The walls may well get thicker just as they got thicker when the trebuchet was perfected, but without flat trajectory cannon mounted defensively, they won't have alot of reason to go lower. And its going to retain the medieval hoardings because its going to have much greater concern with plunging fire and bursts than the people who built star shaped citadels did. Fantasy castle designers are going to design around fireballs, walls of fire, nastiness dropped by flying attackers, dragon breath, and so forth - not around the impact of high speed flat trajectory iron shot and the ability to counter with the same.
And let's not get too dismissive of medieval fortifications. When the 13th century constructed Harlech was besieged by cannon in 1647, it still held out for nine months.
The main overlap between the idea of the star fort and the fantasy castle is that magic is going to necessitate more of an active defence. Fantasy architects won't build castles with the idea that defences will allow the castle to be passively defended successfully. Just as the cannon and perfected trebuchet forced castle builders to start including the means to destroy beseiging weapons in their plans (cannons and other seige weapons on the walls of the castle), so to would aerial attackers and spellcasters force the architect to consider the means to active counter such threats. A fantasy castle must be constructed with the idea of destroying the besiegers, not that you can resist any attack behind your impregnable walls.
For those that want an excellent work covering medieval fortifications, I highly recommend 'The Medieval Fortress' by J.E & H.W. Kaufman.
And, in 3e, take a look at the damage brown mold does. That's certainly a larger threat than to just 1st level characters.
As in the real world, I suspect biological agents will pose a significant hazard to those that deploy them as well as those they employ them against. Brown mold is extremely difficult to handle, and its spores are likely to create numerous hazards when the weapon is stored in peace time.