No, it's not.
Yes it really is. You let the players decide when they start a rest, but you don't let them decide when to finish it. You can't have it both ways - either they get to say they are done or they don't. And with you, they don't. Their decision has to be vetted by you first.