I don't buy that. Otherwise, I can just say "I squint at, and focus on the wizard, to make it look like I am casting a spell in a subtle fashion." Heck, I can make it look like I am starting to perform somatic components. What, are they going to try to Counterspell me?
What it comes down to is...