Actually, the -20 rule makes things worse, not better.
If you're grappled, your only real option is to stab your opponent with a light weapon, or try to break the grapple.
A monster taking the -20 can make multiple grapple attempts, and if it has improved grab, it loses nothing from making them.
Hence it will at least attempt to grapple with each of it's attacks, and will typically have (because of all the bonuses that a grappling creature tends to get over a grappling PC) a 50/50 chance of holding the grapple, WHILE it beats on everyone else. The kraken mentioned above would have been perfectly capable of taking -20 on something like 8 grapple attacks, with no penalty for failing, and a good likelyhood of immobilising the entire party, or at the very least every non-warrior member of it.
As for improved grab being uncommon? I count 69 creatures with it in the MM, out of some 260 creatures. That's 1/5th. As for it being factored into CR, looking at some of the stats on animals, it doesn't appear to be (number of attacks and hitdice appear to be the major influences - for instance look at the difference between a black and a brown bear - double hit dice, double CR. The black bear has no improved grab, but the brown bear has improved grab and extra leverage for it in terms of size)