That's a good way to put it. I'm not sure I've seen the idea expressed that way. (But then I've been avoiding the floating ASI conversations.)My problem is changing the fluff from having that bonus being attached to something that made sense to me to having it attached to nothing. You can attach it to something else. Culture works for me. Level Up does background, which is also fine. Completely untethered ASIs lose all meaning to me. If you're going to do that, just get rid of them altogether; the game doesn't need them to function anyway.
For me, being untethered is fine. The ASIs are just another step in character generation, no more or less attached to the character than to the dice rolls used to generate ability scores (or to the player's point-buy selections).