First of all, there is no initial save for Silent Image, so I'm confused what the original post means by "each of the 3x all failed their spell save DC". Once the spell is cast, the image manifests, and appears to witnesses as real unless they take an action to investigate it using an intelligence check or they physically interact with it. I don't find the DM's ruling out of line, as seeing an arrow pass through the illusion would definitely count as "physical interaction", just as entering melee with the illusion would entail physical interaction. Having the monster roll to attack is (say an AC 11) seems plausible. The illusion served its purpose as a distraction, causing the monster to 'waste' a round's action shooting at the image (assuming it 'hit' the AC set by the DM). Where there might be some room for argument would be if the rest of the monsters noticed in the chaos of the combat or not.
The argument that it appears that the monster just missed the target until they make their investigation check appears to have merit at first blush, I would definitely consider that at long ranges/poor visibility, but this seems to be giving more power than is warranted to the spell. For instance, if the monster had entered melee with the image, would it appear to the monster that he just kept whiffing? No sound or tactile sensation, but the image just keeps moving out of the way at the last instant? Is the caster somehow causing this fine detailed control? If so, I hope the caster is spending an action for this effect as the spell says: "You can use your action to cause the image to move to any spot in range." This would definitely fall under the category of "movement" in my book, though I think it is debatable that this level of control is implied by the statement.
Another example: Say you cast the spell to represent a boulder, tree, or box of some type that the rogue, McStaby, proceeds to hide in with his trusty short/cross bow. McStaby then proceeds to shoot out of it as the monsters approach, for full sneak attack damage due to advantage from Hiding. Great! The monsters may even be surprised. "Hey where did that arrow come from? Somewhere over there! Ouch, Bill is hit!" At some point, the monsters have to notice the arrows shooting directly out of a boulder/tree/box and the game is up. Is this physical interaction? Is the spell basically in place invisibility a level early, due to the fact that many monsters don't have an investigate bonus or an intelligence bonus? Perhaps many DM's would give a bonus to any investigate check made under these circumstances, if not outright rule that seeing an arrow come out of an inanimate object counts as physical interaction.
I guess the bottom line is, as always, go with what works at your table. For new/inexperienced/young role players, it would be nice to give them some leeway for these spells; but for hard bitten min/maxing power gamers, not so much.