After using Viscious Mockery for the sixth time in the last session, my sister just gave up trying to explain how it enraged and caused psychic damage to the grey ooze. Trying to rationalize how power effects work in the context of the game world when it doesn't quite make sense really pulls us out of the game.
I'll offer a piece of advice from a guy that used to have these kinds of arguments at the table prior to 4e when explaining "magic". Just abstract the explanation and you solve the problem.
Using your example of viscious mockery.. the bard may make a lewd comment to mock an ooze, but the magic of it produces the translation and desired effect. It's not for the bard to know why it works and the ooze doesn't understand common.. but the power works because the bard magically ends up pissing off the ooze.
If you try to explain it beyond that, you're distracting from the game and creating "negative time" where players end up complaining instead of being caught in the setting.
I think that I'm a pretty good DM, and I've been doing this for a long time, but I've never had such difficulty keeping role-playing in an RPG. A better DM than I can probably manage it well enough, but I'm starting to feel that it's beyond my abilities.
The fact that you care enough about your game to have a logical explanation for the powers shows that you've got the stuff to be a good DM, but you're falling into the modern DM/fantasy milieu trap. Magic makes magic happen. If your players can explain magic, it's not magic.
Is this the fault of the game system? No, although I think that the game system certainly influences this. Adventure design and my weaknesses as a DM are ultimately at fault.
Another sign that you're a good DM. You're attributing yourself as part of the problem. Whenever a game goes down the wrong street the DM is at least half responsible and usually a lot more.
It's great that many gamers are able to take the 4E system and be deeply engaged within the game world. For some players, such as myself and the members of my group, the crunchy bits require enough attention so as to leave very little left for interacting with the fictional elements of the game. So I can see where those players are coming from; yes, 3.5/PF has a great deal of crunch, but somehow I never found it nearly as disruptive to staying focused on the fictional game world.
I expect that everyone is going to have different experiences that impact their feelings about 3E and 4E. Mine are that I used to hate running combats in earlier versions of D&D. There wasn't enough to keep players involved during combat and it eventually dragged into the following sequence.
1. Roll initiative.
2. Move
3. Roll to hit or Buff or magic goes off.
Usually magic was overpowered, and martial types rolled a D20 and got standard damage.
So what would take the place of a lot of combat in my early edition 6-8 hour games? Role-playing. There was more to do, and more options for storytelling because combat was.. hate to say it.. dull.
So with 4e you see a lot of work done to "game" the combat into something that required mental attention and acted a lot like a game within a game. Combats take longer because everyone needs to think ahead, not just the casters, AND characters have synergies, AND the situation can change from round to round.
(Not to say that the last bit isn't true in early editions, situations did change then as well, but that usually didn't have such a big impact on what players did.)
On Role-playing and skill checks.
If you want to defeat the rolling role-play, make the players role-play the encounter before allowing them a skill check and add a bonus or penalty to the roll based on what they role-play.
If you want to use skill challenges, same thing. Make them come up with a plan that fits the skill challenge, before rolling the skill challenge and be flexible with what you have planned.
If you want to run a role-play heavy game one week, you can do it. Just don't set up a combat encounter and set experience awards appropriate to the risk of failure. If they're going full rp against characters with the same skill levels in appropriate things, then it's a decent base level encounter award should they succeed.
I could go on, but I think I've over-written as is.
Best,
KB