S'mon said:
If you let PC Rogues hide & sneak attack enemies during combat who are denied their Dex bonus, don't you let NPC monsters do the same? I thought it was pretty clear in 3e that you don't get your DEX bonus if you're unaware of the opponent until they attack, and unless you have a special ability you can still lose Dex bonus even if you are aware of them but they're invisible etc.
S'Mon, thank you for saying exactly what I had in mind!
Surprise is mechanically impossible once combat has started (apart from the 'emotional' suprise of course). No argument there.
The real question here is: 'should you lose your Dex bonus against an unexpected attack of this sort?'
The root princinple is that you lose your Dex bonus to AC whenever you are unable to adequately defend yourself. There are many literal examples in the RAW of this principle: when you are flat-footed, when you are being flanked, when your opponent is invisible, when you are climbing or balancing, when somebody performs a feint etc. etc.
Your opponent digging up through the dirt is obviously not among them, but I don't think the examples mentioned are meant to be all inclusive. There are always situations coming up in a game that are not precisely covered by the rules. In this case we need to rely on the principle or the spirit of the rules to make a fair judgement.
Whether or not the ghouls was able to see/sense its target is irrelevant. Maybe he had tremorsense, maybe he was telepathically guided, maybe he just had fortunate timing, whatever. The fact of the matter is: he was hidden from the Sorcerer until his own action.
Obvioulsy, when the Ghoul starting digging up, he was not hidden from the Sorcerer anymore. Combat has already started and the Sorcerer is not flat-footed, he is looking out for danger, danger from all directions, including from the ground.
The only way I would let the Ghoul take advantage of his 'surprise action' is if he attacked the Sorcerer with his very first action. No digging, no moving, no standing up. And even then I would probably allow the Sorcerer a Spot check to notice the Ghoul and retain his Dex bonus unless the Ghoul had readied the attack action.
The same goes for all those weird Hide-in-Plain Sight sneak attackers I keep hearing about. I would no more let a rogue hiding in another room emerge from hiding after combat has started, enter the current room and sneak attack the Sorcerer than I would allow the rogue to hide in the same room (by using Hide in Plain Sight for example) and emerge during combat and sneak attack the Sorcerer. The moment the Rogue moved he becomes visible to the Sorcerer and the advantage of his hiding is instantly negated, it does not last until the end of his own action.
Summarizing: surpise: no, flat-footed: no, losing dex-bonus: maybe, but not if the situation is as you described it