Seems odd to me to limit free actions to once per round or once per kind of action.
Absolutely agree - it should only be restricted as appropriate for the type of action.
So drop something in both hands? Sure. Grip a weapon in one hand or two hands? Sure.
Drop 200 things, one by one? Definitely not.
Change your grip twice (or four, or six, or eight times) in a round to eke in an extra point of damage? Depends. Eight is certainly starting to get silly, so where is the line drawn?
But as a player, I don't see a reason to do it. As a DM, I see no reason to allow it. If a class has a restriction, then using a rules loophole to get around that restriction isn't necessarily good gaming. If you're in a tournament, then you find every advantage you can.
For a home game, making the game easier doesn't necessarily make it more fun, and doing something that is questionable that might cause _any_ member of the table to be less comfortable with the tactic is not worth it. At all.
So, if WotC wants people to free action switch grips to and fro every time they make an attack (and I've made 12 attacks in one round with a character due to opportunity attacks and a warlord - that'd be 24 grip changes), then all they have to do is state it. In the meantime I don't see a need to give swordmages a +1 damage for doing this any more than I see a need to give a +1 damage to fighters using shields because well, let's be honest, they don't need to fully drop their shield to just get a hand on their weapon so all they need to do is grip it _really briefly_.
Obviously, some people think differently.
