Well, true if the whole party (or at least the three you've mentioned) focus on the Pey. But if we take UK's recommendation of the encounter group, the Pey is just one of several monsters in the fight, even though it is the strongest. And being a lurker with constant Invisibility, at least at...