what you should do is have the hand hold the bow string, so it doesn't fire....(because the string obviously weighs less than 20 pounds

)
Honestly, i wouldn't let it give anywhere near an -2 to attack. If mage hand could give a -2 penalty, they'd have said so. Mage hand is for moving things, NOT DEBUFFING.
imagine an at will power that costs a standard action, and a minor to sustain and gives a -2 penalty to all attacks, no roll required. Thats a fairly decent combat power, not a cantrip.
I see mage hand as weak and slow. The fighter/ranger would feel an gentle tug, but nothing he could compensate for, not strong or sudden enough to cause balance problems. You don't have to 'say yes' to making a cantrip into a fairly strong combat option.
'Say Yes' would be applicable to trying to steal the arrows out of his quiver, steal throwing knives from a bandolier, etc...those are fun things that you can do by moving things. You could also 'say yes' to pestering him with pressure to the weapon or body, but it would be narrative "you're annoying him" or "he feels uncomfortable with your spectral hand touching his bum" not "he takes -2 to hit". Fun in game stuff is a great thing to 'say yes' to, players will have a blast, -2 penalties will just make players use it all the time, and that benefits no one.