I'd flip it. Give everyone 12 Action Points. An Attack costs 5 pts at Lvl-1. Every Extra Attack feat reduces the cost by one action point. Thus for a Fighter, at Lvl-5 an attack costs 4AP, at Lvl-11 3AP, and Lvl-20 2 AP.
Similarly, a Cantrip costs 6AP to cast and a Spell 12AP at Lvl-1. If they...