This is a good point. Learning your first language could be as easy as touching the magic item in townIt's a fantasy world, so why not just assume (as the rules say you should) that everyone just learns the sign version of a language at the same time they learn the language? The implication is that that's just what everyone does. This then applies to NPCs, too, so it's not like the players are getting away with some secret code language baddies can't decipher.