Most people miss it because it's not actually stated anywhere ever. Some people claim that it is implied, but that's as far as it goes.
Personally, I wouldn't allow it, since giving a primary caster access to the spell lists of non-primary casters (Bard, Ranger, Paladin) is probably a recipe...