These are my experience with the CR / EL system:
1) Custom Monsters, Monster Advancement
Custom made monsters and advanced monsters are hard to gauge into the system. This also applies to custom monsters from modules our group played. The reason might be the lack of playtesting, but sometimes I also think it is also because people that designed the monsters didn't understand the rules well enough to see the flaws in their design.
To get this right, you _really_ need more eloberate rules or at least guidelines, especially if you know that most module designers and later MM writers don't have the resources to make such extensive playtests as in the base rulework. The experience made when play testing the rules the first time must be turned into a rule system and/or guidelines.
You might say that there are a lot of good guidelines described in the 3.5 MM and later MMs, but there are still limits.
Example: What spell (spell level) and caster level is appropriate for a monster of CR X, if you want it to have spell-like abilities? Some spells are easy (no Fireball CL 5 at CR 1. But what about Blasphemy at CL 20 with CR 15? Why is this different from, say, Cone of Cold?)
2) NPC Challenge and 1 on 4 fights.
The CR/EL rules for NPCs with levels don't work out well, even with non-associated class levels for monster.
Especially single-classed non-monstrous NPCs often aren't worth their CR.
You can't put a Barbarian 14 against a party with average level 10 and expect it to become a tough fight (even if no spellcasters are around). 4 actions per round vs one mean that this encounter isn't really worth more than CR 10. It changes a lot if you add some helping NPCs to get the same EL, but this is not what the system tells you in the face - and what is if you really want them to fight just one creature? Basically, the only answer here is use the same as was done for the Dragons - say your enemy is CR X, but it's actually CR X+3 (and hope that you didn't forget that some high level spells are based on level difference and might turn the whole thing in TPK)
3)
The basic assumption of the DMG that a party will mostly fight enemies with a CR around its party level didn' work out well, either. Maybe it is just because of the omnipresent Wands of CLWs, but unless the characters have to get through a lot of these encounters in a short time frame, they aren't particularly exciting, and thus you typically end up with fewer encounters with CR ranges of L +2 to PL +4, sometimes even beyond. A side effect of this seems to be that a lot of the newer modules or the dragon adventure paths (which so far I have all liked) are pretty deadly (not neccessarily TPK deadly, but deadly nonetheless) which is not always bad, but there are limits...)