Hi! I do have some thoughts taht have been cooking in my head on this, which I'd love your thoughts on.
-Resistances and immunities - this is a tricky one: I do feel that they are relevant in CR at times, but most often their importance is much more negligible than you'd expect. I think I should have gone into this more and it might be something worth expanding further in further posts or publications.
Here's my thinking on why an immunity or resistance is fairly minor:
It's tough to design for all adventuring parties, since you don't know exactly how the party is composed, so I made some assumptions. One assumption is that half (or more) of the party will likely be making weapon attacks, while half (or less) will be dealing damage with energy attacks (fire, cold, and the other non-bludgeoning, piercing, and slashing damage types). There are 10 energy damage types, which means taht, on average, each energy type deals at most 5% of the expected damage in a generic encounter, probably less (1/10 of the 50% of damage not done by weapons). Let's say 4%, since I bet that b/p/s damage is actually higher than 50%, especially at low levels. So at first I'd expect a damage immunity to negate 4% of damage - that is, provide about the same benefit as a 4% increase in hit points.
However, it's probably less than this, since players are likely to avoid less-useful damage types, and most spellcasters have lots of damage type options to choose from. In order to avoid a damage type, they have to know about the immunity, which they probably already know or learn quickly. They may a) be using logic (I bet this frost giant won't take too much damage from my cone of cold) or previous experience (that poison spell didn't seem to do much to the last undead creature we met) or they may even learn during an encounter that a particular damage type is ineffective (hmm, my fireball didn't hurt the red dragon, I better not do that again). Therefore, an immune damage type is less likely to be used on a monster. Let's say that cuts down each damage immunity's benefit by half, down to 2%.
This above is all back-of-the-napkin math and based on assumptions about average party composition and so on, but that's the basis of my thinking. I think that an immunity is probably worth about the same as a 2% hit point boost, or 2 extra hit points for every 100 hit points a monster has. Resistance, of course, is worth half this: 1 extra hit point for every 100 hit points a monster has.
By this math, a single resistance or immunity to energy damage is just not enough to change a monster's CR.
Now if a monster has many immunities, that can build up. A ghost has 3 energy immunities and 4 resistances, which is worth taking into account in CR--especially since, while most spellcasters can easily avoid using a certain damage type, they may have difficulty avoiding all seven of a ghost's immunities and resistances. Using our 2%-1% rule of thumb above, we might say that all these energy protections together are worth about 10% extra hit points, or maybe a bit more since spellcasters might be forced to use an ineffective damage type even once they learn the ghost's resistances. Still, by CR math, a monster needs to gain 30 hit points in order to raise its CR by 1, and a 10% or 15% increase in hit points is not usually enough to do this until the monster gets to around CR 15 or higher. At this point making a rule like "7 or more energy immunities and resistances raise a monster's CR by 1 if it's CR 15 or higher" begins to seem like such a corner case it doesn't even seem worth noting.
Now it could be that my math/assumptions are wrong and the benefit of a single energy resistance/immunity is higher than I realize. But this is my thinking.
Immunity, or even resistance, to bludgeoning/slashing/piercing is different - that is quite valuable, since I think most parties do a ton of that type of damage. And I should have noted that such immunities/resistances are very valuable. But they are rare, and mostly for swarms. It's worth mentioning the much more common resistance: to nonmagic weapons. The value of this is entirely campaign- and narrator-dependent, so to figure out its worth we need to decide how common magic items (and the magic weapon spell, etc) are. I think that in a lot of campaigns, from character 1 to maybe 5 or 6, magic weapons are nonexistent or rare, and resistance to nonmagic weapon damage is very valuable - worth, perhaps, as much as 50% more hit points at CR 1, and decreasing rapidly. By the time you get to, say CR 7, I'd think that it's of almost trivial benefit in many campaigns. (But then, what do I know about most campaigns? I don't have as much data as I'd like.) If I were to revise the monster creation guidelines, I think I would note the value of resistance to nonmagic b/p/s damage at low CR.
-Number of monsters: why doesnt encoutner building take number of monsters into account?
In the 5e DMG, a number-of-monsters qualification was added to their formula, adding a multiplier to the monster's total XP if there are a lot of them - which was necessary in that system, since their method of encounter planning - totaling XP - is exponential and undervalues low-CR monsters. Since we use a totally different way to plan encounters - adding CR, which is more linear - we don't need to use this multiplier as a fix. We already believe large numbers of low-CR monsters are much more dangerous than the DMG does! (Also, WOTC seem to have dropped this multiplier in Xanathar's encounter building as far as I can tell.)
