That's less than half an answer at best. Clerics get their spells from their gods and warlocks from a patron. They have an eternal trainer baked in where "go find someone else" makes no sense. The assumption would be that they are already worthy of the spells. (And if you are using Milestone leveling, that should be baked into the plot)
And if the patron/God finds the caster unworthy of higher level magics, force them to multiclass (see thread "I think we can all agree multiclassing is broken....")
The other side are Sorcerers who "awaken" their spells from their bloodline and bards, who figure spells out from everything they encounter while leveling. Magic Secrets is pretty much "decode other classes spells on your own" except now you need a teacher for your own class spells? Or your draconic heritage which lets you grow wings falls down on mastering spells? That seems nonsensical.
Imo with wizards, the level-up spells are fine, it's all the rest that should be hard to get. The "wizard with a book of all spells" is the real pain point.
I say this as a player with Ritual Caster (Wizard). We don't have a wizard and I have spent 2 levels searching for Contact Other Plane, and I am fine with it. (Mildly irked if I don't find it this level though)