Remember that certain assumptions went into the CR system, the same assumption that went into the entire creation of the 3e core rules.
The assumption is high magic. If you're playing low magic the CRs will be off, way off. However so will then entire hit point system. If you're playing low magic I recomend grim -n- gritty combat rules they work nicely for low magic.
You also have to take individual party abilities and tactics into account. If you've got a party walking on the planes and Behir sticking out it's head and looking at the party is much, much more dangerous then the party coming up behind that same behir. I fing that house rules for modifying CR are always in need.
For example, my party has repeatedly done horribly against gaze attacks, so if a monster has a gaze attack it's CR is instantly +2 but I don't recommend that others do this. Also my party seems to make light work of monsters who use flight tactics so those get a -2 CR. This keeps the system balanced for me but it probably won't for you. I like the CR system but as with the rest of the core rules it needs to be used as a guideline and not a holy grail.
Another point that was made was XP. What I do is I use the monster's treasure listing. I calculate the monster's treasure and then half of that becomes XP at the value of 1gp = 1xp. Works pretty well for me slows down wealth and level gain. If the monster doesn't have treasure then I just assign XP arbitrarily.
I like the idea of assigning tasks a CR though I think I will do that, it will help me calculate how much gold and xp to award for things.
All in all this is a good thread and a great discussion on the possitives and negatives of CR, DR, EL and ECL.
good show.