A beast might kill a familiar if it sees one. Again, why isn't the familiar sneaking? Moreover, there are some familiars that many beasts won't tangle with. A typical domestic cat is not going to attack an owl, for example. The owl is too big to be appropriate prey, and the risk of injury is high. Don't let their appearance fool you, owls can be surprisingly vicious.
A familiar might be sneaking, but a +3 is not that great and a low check will ensure he is seen, even an average roll will fail many times. You can make him invisible, but that won't give him advantage on everything.
I think a domestic cat is not the kind of animal you are scouting for. If there is a domestic cat in the dungeon it is likely a pet of someone or something and in any case is largely irrelevant, your paerty is not at risk for strolling up on the cat either ..... unless it is a familiar of something else in which case the Owl is at risk of exposing you. I was talking about beasts that would be encounters in areas you are scouting - more like a giant constrictor snake, sabre-toothed tiger or dire wolf.
Intelligent creatures might kill an owl. Also only if the detect it. Why is the familiar not sneaking? If this is the issue, however, why not use a less conspicuous form? A rat or spider would hardly be out of place in a dungeon
I mentioned that above, but this is an hour to summon (more as a ritual) and it it an hour to summon your owl again. The downside with either of these is they are pests and will probably be killed for being pests.
You would need to summon something that is both common in the environment and not considered a pest. Indoors that is tough.
. If the dungeon denizens plan to keep their dungeon 100% pest free, they've got their work cut out for them.
There are very few animals deep in caves beyond the "living cave", moreover that doesn'tmean they won't. I can't keep my house free of spiders and I can't keep my garage free of mice, but I darn sure kill everyone I see.
Also, the familiar only disappears if killed. Does the familiar not try to flee when detected? I would think that the familiar would win initiative at least some of the time. An owl familiar, in particular, has flyby attack and a fast movement rate, so it should have a good chance to escape even if it is detected.
If it is detected and not suprised and can escape and they do not have ranged weapons. It depends on numbers and environment and a lot of other factors.
Again, a lot of the issues you bring up with the familiar would also be a problem for the rogue. Larger beasts will attack him. Enemies who spot him will attack him and/or raise the alarm. Heck, the rogue is at least as out of place in the dungeon as an owl, and significantly more so than a spider or a rat.
A Rogue is generally going to be a lot better at sneaking, will have a higher initiative, can survive one round of combat and with cunning action is going to be able to outrun any familiar and the vast majority of enemies.