I had never considered this, but I really find the idea appealing.
This would assume the notes are comprehensible to anyone or clearly marked for their purpose, and even then, if they end up composing a significant product.
It's certainly a myth that he only wrote for himself; he said in discussions here that there were serious differences between what he created and what he made for himself.
I think a detailed Castle Greyhawk/Zagyg that's sitting around, waiting to be published is probably a myth. What seems more plausible to me is some reconstructed version of it, whose value as an insight into what it really was will largely depend on the process by which it was created.