First, notice that the wording is quite clear: "Characters who turn their attention to other tasks as the group travels are not focused on watching for danger." They are not focussed on watching for danger does not mean that they are completely oblivious to their surroundings. It's not specifically their task, but they are seasonned adventurers, and survivors.
Now, the only thing that the rule says is that the passive perception does not contribute to the group's chance of noticing hidden threats. Since it's about the group, switch to the group activities, what does it say ?
- "To make a group ability check, everyone in the group makes the ability check. If at least half the group succeeds, the whole group succeeds. Otherwise, the group fails." That what it means "contributing to success, having a roll that matters in counting the result.
- Then, there's the example: "Group checks don’t come up very often, and they’re most useful when all the characters succeed or fail as a group. For example, when adventurers are navigating a swamp, the DM might call for a group Wisdom (Survival) check to see if the characters can avoid the quicksand, sinkholes, and other natural hazards of the environment. If at least half the group succeeds, the successful characters are able to guide their companions out of danger. Otherwise, the group stumbles into one of these hazards."
Now, translate this into detecting an ambush by orcs. It's exactly the same thing, if enough people of the group succeed, they will know about the ambush
as a group, and therefore no-one risks being surprised, since everyone is aware that there is a group of orcs ahead. But if they fail, either because their scores are too low, or because some people are not focussing enough on the threats (and their passive score is simply not counted as per the rule), then the group is not aware that there is an orc ambush ahead.
So when combat starts, everyone will check for surprise, using their passive perception, which has not been magically turned off, simply not counted in the group result.
It's mostly about timing and determination for the group, since it's travelling and it's focussed on the group, not on single characters.