RAW, Mirror Image simply says that if they are able to perceive you with senses other than sight, specifically mentioning Blindsight or Truesight, then Mirror Image has no effect. I’ve read where some people believe that by being grappled, they are “perceiving” you through a sense other than sight, I.e. touch.
But I’m not convinced that would work, if I were grabbing you and three other me’s were, too. Sure, you’d only feel the one, but people rely on their eyes a whole lot, and in the confusion of a fight, I think it is probable that the target could swing a punch at the wrong face.