To tangent completely off what I was talking about before, since I really wasn't comparing the two, here's what I think are the divides between XP and Milestones. I'm sure this isn't comprehensive and there are more.
1. What they are given for. Milestones are usually given for completing, well, milestones. The party could "skip a lot" and still be given full XP. On the other side, XP is given for killing monsters, which means things like using resources to avoid an optional battle is a losing proposition. It engenders a "every battle that can be joined should be joined so we can level sooner" attitude.
This is a good question. I do very much like the idea of an adventuring day. PF1 is still my fantasy RPG of choice. I design my adventures to be completed with a minimum of encounters, but with a higher total possible. For example, lets say a 5 room dungeon with a swamp that leads up to it. 12 encounters total. A very savvy group with good luck gets through it in 6-8 encounters. An average group playing conservatively gets through it in 9-10. A fumbling group with bad luck encounters all 12. You would think folks would intuit that completing your goals in fewest encounters would net the best results. However, I found that to my players more encounters equals more XP and treasure so all 12 would be painstakingly encountered. Even to the point of retreat, rest, and return. Even if the goal was met, they would return to uncover every possible encounter!
I also like to put forks in the road and give the players challenging, but interesting decisions. Go right and stop the hobgoblins as you were tasked to do, or go left and save the princess which you just found out is being kept in an old abandoned keep. Now, the choice is between duty and the seemingly right thing to save a defenseless person, but not to my XP hunting players! They would have a long conversation about how "convenient" the princess happens to be. That, surely its a trap of an area too high level to handle. Surely the best route is to take out the hobgoblins so that the party can score enough XP so they can then handle saving the princess. Duty and/or honor who cares?
After the above experiences I realized that XP was just interfering with what I wanted to accomplish as a GM. I want to put forth a challenge that is solvable in smart and unique ways, I dont want to produce piles of encoutners for the players to charge their characters into face first. I want them to act like actual characters in a story trying to resolve issues and face dangers seriously. Secondly, I want the players to think as their characters and be driven by their morals and reasoning. I want the RP in front of the G where the character matters and is not just an avatar to move around the board. (In addition, I find treaure for power exhausting as the players strip out the copper wire and plumbing of every dungeon for an extra gold piece at the end.)
Removing XP and going with milestone was the answer. It took a little time, but eventually the players began to focus on the narrative and goals before them. they also started caring what their characters thought about the situation and acted on it without fear of being behind or screwed out of treasure. It was much needed breath of fresh air and life into my RPG experience.
2. Individual vs. Group. When people say Milestone, it's 95% of the time short for Milestone Leveling, which is just giving out a level. With that there's no reasonable way with that granularity to track individual contributions and/or attendance with XP bonuses, or even replacement characters being a lower level. (Milestone XP, where milestones are much more common and grant XP, avoid that, but are not a commonly seen tool.) In something like a West Marches game where player have a stable of characters they really need to be able to track individually.
This one is more mechanical for me. I put a lot of effort into bringing my games alive. Worrying about every piddly XP point collected was just cognitive load wasted in my opinion. That was one less interesting map, or one fewer NPC to play, or a less interesting intriguing mystery placed before the players. Also, having mixed level parties was a total PITA. I either made the encoutner challenging for the top level player, thus making mincemeat out of the lower level ones, or the whole thing was a cake walk. The mix was like 1/3 right, 1/3 too easy, 1/3 TPK in waiting. With everyone on the same level, encoutner and adventure crafting bcame much more managable. (Interesting enough, PF2 made mixed level parties suicide by design!)
I really dont care if Billy comes late, or Suszie isnt brave enough with her character. I dont need to bait effort out of my players with XP. The rules are right before them. They pick locks because thats something a rogue can do. They convince NPCs because thats something bards can do. They kill tough monsters because thats what fighters do. etc... The mechanics are the toolkit for the players to engage the setting, the msyetieries, the political intrigue, the exploration! Lastly, I want them to work together and not worry about who gets what. Its a team game so group XP it is, and even better with milestone!