My own experiences with the CR in 5e has been mixed. In a short campaign where I was a player, the creatures seemed to be pushovers. Our party was regularly taking down creatures that we had no business taking down at our low level.
Once I started DMing, though, it's been consistently 180 degrees opposite that first experience. I recently had a TPK (I don't like TPKs, I don't set my combats up to be inappropriately difficult, and I attempt to, without the use of any sort of Deus Ex Machina, keep at least one party member alive. This time was impossible for me to see a solution to) where the party wiped to kobolds. I mean seriously, kobolds. Prior to that the party was very nearly wiped by a Gnome village they decided to attack. After that, in a new campaign (post-kobold TPK), the party hasn't come close to wiping, but every single fight has so far had at least one player go down (not dead, just down). And we're talking CR 1/2 zombies and the like here. The fights have actually, each one of them, felt pretty epic, with lots of movement and burning of churches and the like. Last night's game involved a Bullywug chieftain damned near killing the party's rogue (the player rolled a natural 20 on his final death save. He got back up, dusted himself off, and demanded the party let him take the bullywug's armor, or else. And legs. Frog legs is good eating. Talk about hitting a grand slam on a full count. Man, epic).
So, I know, wall of text. But I really haven't been able to wrap my brain around the difficulties yet. I don't know what I as a DM am doing differently than the previous DM. I just don't get it, from an adventure-creation, this-is-what-the-difficulty-should-be standpoint. So for now I'm just winging it.
EDIT: Actually, I think I do know what the difference is. That previous DM regularly tossed very tough creatures at us, but they were all solo creatures. My own DMing style is different, in that I tend to toss in a lot of soft squishy creatures.
So, in 5e, the more critters you have, even of low levels, is going to create a MUCH greater challenge to the party, whereas a solo one-shot is just going to be a pushover, even if its CR suggests that it should be difficult.