That sounds workable, but I’d probably just say the Spellcasting must still make noise, and that part of learning a spell is essentially designing your own version of the process of casting, so maybe a silent character has to clap or snap in specific sequence, or can wordlessly shout and hum, or...