I just house rule that every encounter is a milestone. Simplifies it for me and increases the number of action points.
I find that making every encounter a milestone actually rewards them more for continued play. Your action points reset to 1 at the end of an extended rest, and the players like to accumulate action points for harder encounters.
Agree with you, ignore milestones if you want, keep them if you like em. But if we're discussing the merits of RAW, I have to say I think they should have left them out. Awarding more action points is fun and more simple, and doesn't seem to unbalance anything.