Yeah, it seems this encounter can't be lost. It would inevitably move backward until all of the guards disengage. I suppose that is by design, so the encounter doesn't go on indefinitely. I mean, the Players could do weird things like stop attacking, let their opponents push them to full hit...