The closest I can find to your interpretation being a general rule is Mike Mearls saying he'd allow it. Note that according to the Sage Advice Articles, only Jeremy Crawford's rulings on twitter or on the D&D website can be taken as official. If you run it that way at your tables, go ahead, the game probably won't break, however I'm going with the interpretation that 173 refers also to class features that let you add your proficiency bonus to a skill, and you have to be explicitly granted permission to change proficiency from a class feature.
EDIT: tweeted Jeremy Crawford. We'll see what he says.