Non-standard houserules aside, that's not the issue. Issue is low level spells may as well not exist, since you use your spell points to cast your highest level spells. Under standard rules you have 4/3/3/3/3/2/2/1/1 spells at 20th level. Under point buy you have 0/1/0/0/0/0/0/0/10 spells at level 20.
That's not how it works. You aren't allowed to create more than one spell slot of each level above 5th per long rest. This actually means you have less high level spell slots than the standard rules, because you never get those extra 6th and 7th level slots.
Yes, you can convert the rest into all 5th level slots if you want. But there are plenty of useful spells that either gain little from upcasting, or just aren't worth the resource cost. If you only need to make one scout invisible, there is no reason to upcast
invisibility.