Wait, wait, wait! Earlier you were adamant that in your games you never have things to be in certain locations and happen at certain times just because the PCs are there then, and doing so is dishonest illusionism, deceiving the players and railroading! Except now you're saying that you're doing exactly that!
"If I can come up with something interesting" requires that it be grounded in the fiction. If I can't ground something in the fiction, I won't do it, flat.
I have explicitly established that the desert, for example, is stalked by dangerous monsters. There's a tradition (it's too loose to call it an "order") of monster-hunters who stalk the wastes, generally filled with adrenaline junkies (Grim World playbook, called "Slayer") and those who are in tune with the land without revering it
per se (Rangers, rather than Druids). This foundation, established literally from before session 0 (due to arising in the
previous group's session 0), gives me reasonable latitude for encounters in the desert, even without any other considerations.
Any time you wander off the beaten path, you're taking a risk of running into Something Nasty; it's part of why trade routes are well-established and have regular caravanserai locations along them (along with both mercenary and city-state soldiers patrolling them for protection).
Those other considerations include the fact that the party is hunted by no less than three distinct factions who are heavily active in the desert: the Zil al-Ghurab/"Raven-Shadows," a mystery-cult of assassins whom the party crossed on their second adventure; the Shadow Druids, a hive-mind faction of druids who revere death and decay and seek to transcend that cycle, whom the party repeatedly interfered with during their second and third adventures; and the Cult of the Burning Eye, a group that apparently worships a Lovecraftian horror imprisoned on the PCs' world and which uses blood magic rituals to drive themselves to frenzy, whom they have foiled two or three times now on different adventures. (There's also a black dragon gang, but that's almost exclusively operating within the main city, and thus largely irrelevant for wilderness encounters--but quite relevant for in-city encounters!)
This is what I mean by saying it isn't hard to make the justifications for this stuff. It really isn't. Just do a
little bit of early work, and inform the players about these things, as stuff their characters should just
know about the setting. Boom, instant explanations for a variety of expected encounters. I mean, I already said I was okay with random encounter tables, and that's
by definition "placing something" in the party's way--but in an ecologically- and narratively-consistent way, assuming your random encounter tables are actually good and not crappy.
I honestly do not see why this is such a "gotcha," though I would be lying if I said I didn't expect you to come along at some point with a message pretty much exactly like this one. I'd hoped not, but...well. Here we are.