Some very nice numbers here for damage output, which makes me wonder - how is it that this idea of "grind" came about? Is it due to the very large HP totals of opponents, such that despite doing 30+ points per hit it will take 20 or 30 hits to bring the BBEG down? Or is there some other factor? Or perhaps the idea of grind is a fallacy?
I can't really answer that. Our group almost never experiences grind combats. It does happen on occasion, but not that often. When it does its usually with a solo that just isn't that interesting. But I've gotten better at spotting those coming, and WotC has gotten better at not writin them. Additionally, our group has gotten better about actually using our powers, rather than hording all of our dailies for a super battle that never comes. The informal rule around the table is that everyone should use about one daily ability per battle, particularly if you're from a class that has a couple stances or a couple sustainable powers that don't work well together in the same fight.
A factor I didn't really consider in my initial query was the versatility of character design. I realize that the above replies relate to my initial query on damage output; what about non-combat abilities? How hard/easy is it to accomplish teleportation, or create an illusion, or charm a target to become your friend? Is that all the type of stuff done with rituals? What about magic items, do they provide a decent level of variety of effects?
It depends on the non combat ability. Some are present, some are rituals, and some have been intentionally removed for balance reasons. Just looking at the ones you mentioned,
Teleportation is pretty freely available for short distances and in combat teleportation. As for long distance teleportation, rituals. My feel is that this hasn't changed that much between editions, but then I didn't abuse teleportation in 3e. So at least for the casual user, its about the same.
Illusions are pretty freely available in combat forms for the right classes. Stuff like making the illusion of a vast pit appear beneath an enemy is now considered to be an explicit combat action rather than a combat application for a non combat ability, so you'll find it in the combat powers for the appropriate classes. Beyond that, rituals. It depends pretty heavily on what sort of illusion you want, but rituals is where you should look.
Mind control has been toned down a lot outside of combat.
Additionally, how prevalent are non-damaging effects in combat, things such as charm or compulsions that can change the behavior of a combatant?
Pretty common, although a lot of them also do psychic damage. Like take your typical attack that forces an enemy to attack one of his friends- its going to probably do some psychic damage to him first, and then force him to attack.