I tend to agree with most of the others that have posted, but I want to say that I understand the opposing viewpoints. Clearly there are no explicit rules about how to deal with AoOs against invisible creatures, so you just have to house rule it (at least for now) and do what you and your players think best.
I think it makes sense that in order to get an AoO, you have to know that an invisible creature is nearby, be able to pinpoint its square, and be able to perceive somehow the event that triggers the AoO (hearing a spell being cast, hearing the creature drink a potion, etc.)
I started out thinking that pinpointing the square is important because the AoO is supposed to reflect perceiving a weakness or lapse in the enemy's defense and launching a quick strike while their defenses are down. As I wrote this paragraph though, my mind wandered to the other side:
Suppose I know an invisible creature is near me because I heard it or saw it but I'm not sure exactly where it is. Probably I've narrowed it down to three squares as I probably have a vague idea of whether it is either in front of or behind me, or whether it is on my left or right. I would say that a successful listen check can probably narrow it down to three squares.
If I hear the invisible creature drink a potion or start to cast, I can reasonably assume that an invisible creature is near me with its defenses down, so I can launch a quick strike where I think the creature is. If I guess right and I beat the miss chance, I hit. If you are willing to possibly waste the AoO because you may guess wrong, so be it.
One of the most important aspects of D&D is its abstractness. As for the casting vs. casting on the defensive, the player should be told what the situation is. If the invisible creature has done something to lower their guard that the PC can perceive (drinking a potion, spellcasting), the PC should be told.
In describing the action, if the party is all accounted for an it is known that an invisible creature is present, and if it does something that the PC can detect as a possible AoO trigger, the PC will probably launch a quick strike where he thinks the creature is hiding. If the creature did not actually provoke an AoO, then the PC's attack does not hit no matter what the roll was (since the creature's guard really was up), and the PC is not charged with making an AoO.
Of course, in game terms, you can't just attack whenever you want and you only roll to hit at the specific times you are allowed. But what this abstraction represents is a constant series of blows, with to hit rolls representing whether any of them got through the target's defenses. A low level fighter may swing his sword 5 or 6 times in a round but only one is of sufficient quality to have a chance to hit and do damage during that time. As characters reach higher levels (higher BAB) the character is able to turn more of those 5 of 6 blows into quality chances to hit for damage (iterative attacks).
If the character thinks there may be an AoO but there isn't, the strike falls into one of the blows that is not a quality blow that has chance to hit for damage (because the creature's defenses were up after all). If an AoO is actually provoked, then the creature's defenses are really down and the hit counts as a quality hit and has a chance to do damage.
Does this make sense? I'm interested to hear how people rationalize this.