As someone who had no particular fondness for species mandated ASIs, but also likes attaching them to backgrounds (would have also probably been fine with cultures) more than them being pure free floating, a huge part of the reason why is that backgrounds aren't innate, at all, at least not how I most commonly hear and use the term in this context. Innate, to me, is even if you change most of the circumstances of your life, this part still expresses largely the same.
Likewise, I don't see any connection between having a different starting point in life and the idea that a person could "never better themselves." Nothing about a background stops a character from working to improve any part of themselves.