Ya, I can see the logic behind losing action points not spent.
It makes me laugh if I think of my players who haven't spent theirs and are going to level up.
They're going to want to do something crazy/silly just to use them up even if the adventure is over. I'm sure it will be something...