One the of the best bits in 4e was "Unaligned". To me, that was a fantastic addition to the game.
The whole caricature of chaotics will always lie is no different than saying all evils must kill everyone they see. It's a poor interpretation of the alignment.
However, the notion that a bunch of Chaotics will get together and be able to cooperate long enough to build a society is a bit too much for me. Any long term (as in generations long) society has to lean strongly on lawful concepts - the passing of power from one person to the next is a major element, just for a start.
I can see chaotics working in small, family groups. But a Chaotic good nation? I just never could wrap my head around it. Chaotic Evil? Sure, no problem. There's no shortage of despots in history. But a nation which eschews formal laws but still manages to persist over generations? Just not going to happen.
It's not that chaotic is bad, per se, but, chaotic just doesn't play well with others.