Explain how you're violating the license by saying "no." (Obviously, the person can laugh in your face if you say "no," or just simply ignore you, but I'm not sure how saying "no" actually violates anything other than standard good sense.)
Actually, I can think of an example where it doesn't even violate good sense. Say you come to me and let me know, as a courtesy, that you're going to be using my OGL material in a game about abusing kids. In such a case, I would try to convince you not to do that.