CR should be determined by an encounter that would push a party to its limits- basically having a 50/50 shot at winning.
Why?
I never understood why they chose an encounter with a CR equal to the party's as depleting 20-25% of their resources. It seems like a completely arbitrary number, especially since what comprises 20-25% of a party's resources will vary greatly from party to party, but an encounter that pushes them to the make or break point will be more uniform (though still not perfect).
1) The 50/50 chance is arbitrary. The 20-25% was based on research that said most groups prefered to have about 13-14 encounters per level, and thus diseminated into the XP system.
2) 20-25% of resources will not vary greatly form party to party. Just stubtly. Following the guidelines on treasure and XP awards, it can be resonably determined what the 'resources' of a 'typical' party will be.
3) What makes a 50/50 chance more uniform?
The 20% number is an unrealistic number because it is counter-intuitive and difficult for a DM to estimate easily. When you design encounters, do you design them to be even CR with a group (a cakewalk), or more challenging?
1) 20% is not unrealistic or counterintuitive or difficult for me to estimate easily. About 4-5 encounters between resting periods, about 3 'periods of rest' per level.
2) My group tends to like a faster pace of level and resource gaining, so I design the encounters to be more challenging, sucking up 50-100% of their resources per encounter, with only one or two resting periods per level. Effectlively, this churns out to 1-3 encounters per level. Since my characters gain levels about five times as fast as the average party, I have to mutliply the treasure by five, too.
Doesn't assigning a CR based on an even-odds fight make more sense than trying to figure out how many encounters of a given CR would deplete 25% of resources?
No, because most parties aren't fighting one or two battles per level, they're fighting around 13-14.
(stuff about the point of CR)
CR's are an approximation of a monster's toughness first and foremost. The toughness of the monsters you fight compared to your own ability to handle that toughness dictates how much XP you gain, which therefor dictates how quickly you advance.
CR isn't used for calculating XP directly -- the toughness of a monster vs. your party's touchness is what's used to calculate XP directly. CR is a numerical represetnation of a monster's toughness that can easily be plugged into a formula to determine how much XP something of that toughness is worth.