If I were creating an open world, sandbox style situation, first I wouldn't worry about how badly your players hammer everything since they are likely to fight everything with full resources every day. I still remember my PF1 players going through Kingmaker and my 5E players doing Out of the Abyss, they hammered every sandbox style of encounter they faced because they had no need to preserve resources.I am looking at the encounter building rules and they are based on Party Level. If I want to create a more open world, sandbox style situation, what is the best method?
The thing that seems easiest is to assume any given encounter is rated as"Standard" for it's level. So, an encounter of 4 4th level monsters is a Level 6 encounter if I am reading the math right. If the party is 5th level when they engage the encounter, it is considered Sever for them, but if they are 8th level is is considered Trivial. Actual XP awards for the encounter would be based on their level when they engage it.
Am I reading that math right?
Now that being out of the way, a single equal level challenge or two with some lower level minions is generally a pretty good encounter for a standard 4 or 5 person party in a day. They won't have a tremendously difficult time beating them unless the monster has some kind mobility that allows them to protect themselves from direct attack given the short range of most attack capabilities in PF2.
A single monster of +2 to +4 would be a very tough challenge for most parties.
Yes, a group of monsters 4 levels lower would be pretty trival for a party unless they had some kind of special attack that was difficult to counter like all being to fly and launch breath weapons against a party with minimal ranged ability. You should think about that as well. If you had a group of flying gargoyles with bows that were 4 levels lower against a party of melees with casters using 30 foot range cantrips, then obviously the challenge is going to be much harder than it appears on paper.
XP is based on the level they engage it, yes.