Ratskinner
Adventurer
It can be done...but your no longer playing 4e at that point. You’re playing a severely drifted, serious hack if 4e. And due to the amount of drift/hack, you can’t juat “toggle back” to all things 4e. Well, I mean you can, but the play experience would be jarringly incoherent.
I've had similar experiences when trying inject more Fate, or other "indie" rpg mechanics into D&D (any edition). Very quickly into the exercise, I get the impression that its just not worth the effort, because the real answer is just to switch the other game's randomizer to a d20.
To address to your thoughts specifically WRT 4e; I think that one of 4e's design/presentation paradoxical weaknesses was that the combat engine worked so well almost entirely independently of any of the other mechanics. I think this allowed many groups (at least the ones I know from experience) basically ignoring the rest of it. In particular, I point to the original presentation of skill challenges. I was directly involved in three separate groups immediately as 4e came out.* All three quickly abandoned skill challenges and basically ran 3e games with a 4e combat engine...and then abandoned 4e. I'm actually into tightly framed games, and it totally whiffed for me that that is what 4e was shooting for. This is one reason that I advocate that combat and non-combat scenes should resolve using symmetrical mechanics.
*My son and his friends also had a 4e group full of new teenaged players. They suffered less than the groups full of experienced players I was running with, but they to seemed to whiff on the non-combat aspects of 4e play and using them as you and the other 4dvocates around here do.