I like the speed of play, bounded accuracy, backgrounds, the reduction in needless complexity, the "old school" feel, and the excellent production values. In particular, I like that the 5e PH and MM are books that I actually enjoy reading and flicking through. The 3e and 4e PH and MM were more "rules-based". I referred to them during the game, but I was less interested in simply reading them as a book, if that makes sense.
More than anything else, though: I love that they put the power back in the hands of the DM. Magic items aren't listed in the PH. Some rules are left intentionally vague, and open for DM interpretation. I recognize that some people may see this as a bug, rather than a feature (organized play, etc). However, I have never considered D&D to be a competitive or highly regulated sport. For me, it's a story-crafting hobby to be shared amongst friends. A more flexible rule-set helps achieve this for me.
...
I don't like some of the art. As many have noted, I don't like Driz'zt as poster boy for the elves. Some of the halfling art is questionable. I would have liked to have seen even more humor in the art; I really like the modrons in the MM, the simple line drawings in the MM index (otyugh+outhouse, flumph+10'pole, etc), and the drawings for the Conditions in the back of the PH.
I would have liked to have seen some more classic vistas or maps. I want to see a big full-page picture (+ exterior map) of the Caves of Chaos, or the Ghost Tower of Inverness, or the Palace of the Silver Princess. A thousand adventures have been written and run simply because TSR included a single-page map of the Haunted Keep in the old red box Basic Set, or the sample dungeon map in the 1e AD&D DMG. Nothing inspires a DM (and many a player) like a good map.
Rules-wise, I'm actually pretty comfortable with what I'm seeing. I'm DMing a Druid of the Moon, and I don't feel like he's either over-shadowing the rest of the party, or that he's invincible. Sure, his brown bear form can tear through "standard" opponents, but there are too many threats in my DM arsenal that he's no better at handling. Flying opponents, missile weapons, creatures resistant to non-magical weapon damage, etc. It's a bunch of bonus hp, but he has terrible AC in bear form. I'm okay with it.
Also, you'll never convince me that spellcasters are living in the shadow of the fighters. That has absolutely not been my experience with 5e so far.
The only thing I've had to house-rule so far is the paladin's Lay on Hands. Example: a 6th level paladin has a 30hp pool of Lay on Hands. If that's portioned out at 1hp per time, that's 30 "kick-ups" per day. A heavily-armored paladin can keep the rest of the party in the fight almost indefinitely, better than any cleric. Every time a PC drops, he'll be back up again almost immediately. 30 times per day. If you have two paladins in the party, they can even kick each other up. I've had to house-rule this so that the paladin, if they use their Lay on Hands on an unconscious target, must use their entire remaining pool (rather than simply 1hp at a time). So, that's my main rules annoyance so far (and easily house-ruled).