It's notable that fortresses were still being built in the 19th century, designed with sloped walls to take account of gunpowder weaponry (the German invasion of Belgium in WW2 started with the assault on the fortress Eban Emael, which fell after bitter fighting & heavy losses). Flight definitely changes things though; I think the only real defense vs magical flight is a high-level Greater Dispel Magic zone over the castle's airspace & around its moat, maybe with a narrow non-DM tunnel leading to the gatehouse to avoid ruining potions etc of legitimate visitors. IME non-magical fliers attacking a castle are far rarer than Wizards & Sorcerers so I think DMZ is the way to go for any serious defense.
Basic infantry forts designed vs orc hordes probably don't need these elaborate defenses, though.
I agree w Celebrim's point that a castle should have its own PC-style "rapid response team" with arcane & divine casters, rogue and several Fighter types. If the Lord is 9th level, a team ca 4th-7th level would be about right.