Well, those are two separate issues, really.
The first issue, namely that of divinations, is a mixed-bag. They have oodles of limitations to them, and wise NPCs learn to take adavantage of said limitations. One of my favorite examples is from Piratecat's story hour: an evil NPC taunts the players, by possessing a zombie. The NPC identifies himsefl by name, makes them an offer that they refuse, and then departs. Not much later, they choose to scry on him, intent on giving him what for. The trick? He lied about his name, so the scrying reveals a completely different person...a mutual enemy, in fact. Great stuff.
Another way to make divinations less effective, without using high-powered tactics, is diffusion. Against a single target, divinations are highly effective. Against a large organization of loosely affiliated cells, not so much. Using trickery and discretion, you can prevent your players from asking the proper questions in the first place. A simple alter self spell can cause the players' divinations to be targeted in the wrong location. Commune, in particular, can be difficult to use if you set things up properly. With mostly 'yes' or 'no' answers, it's difficult to use if you don't know the right questions to ask. Commune confirms or denies your suspicions...but it doesn't give you information, per se...merely validates it. If you have ten suspects, Commune makes it simple to determine the murderer, by the process of elmination. Make it a crowded ball with 200 people in the crowd, and disassociate the killer from a direct relation to the victim, and you're starting to get there.
However, as PC's story hour illustrates well (and mine, to some degree, too), the focus changes as the party gets more powerful. The question no longer becomes "Who?" as much as "How?". They know that Lord Robilar killed the duke...but as city councilman and head of the merchant's guild, he's virtually untouchable, for example. We cast commune and
know he's guilty....but the ruling priesthood of Rao says that we are unlawful heretics, using tainted information from Olidamarra that cannot be trusted and is surely some form of trickery.
The second question is part of a different problem: which is how to justify why a vastly more powerful foe wouldn't just smite a low-level group of adventurers? There have been whole threads on the topic, but the simple answer is that such concerns are usually both campaign-specific and usually out of the players knowledge until late in the game.
In fact, you hit on the classic answer in your own post: because there are other factors involved. The beholder-guildmaster may be bound by specific rules in the guild charter...rules the PCs are completely ignorant of, even though they benefit from them. An example like Elizabeth invoking the right of Parley in Pirates of the Caribbean, for example. Perhaps the beholder has equally powerful enemies of his own, and can't afford to attack the players, or give his enemies a justification to come after
him, at least until he's better defended. Sure, I could kill that pesky band of adventurers that revealed my counterfeit Illithid skull manufacturing operation....but that might get the attention of the Illicit Magical Appropriations Bureau on Mechanus. Next thing you know, we could all be up to our armpits in Inevitables. No, just dispatch some brigands not directly related to the guild. make them cheap, stupid and brutal. You get the idea.
Finally, split the party's attention, and you split the resources. Someone close to the party is murdered...but who did it? If the party (who is apparently unpopular) finds themselves with few friends, they may have a hard time deciding
which enemy has come for them this time....especially if it's a new player in the employ of an old. If they waste half of a spell determining who isn't behind the attack, that begins to add up.
I know my party has relegated Communes to the back burner, because their situation is too complex for it to be too useful. The complicated interconnections between the bad guys politics means that the players can never be certain how many dirty fingers were in any single pie...without spending plenty of x.p..
