Difficult, but not necessarily impossible. You appear to be leaving off the requirement that the hidden creature not be heard. There's two criteria here. Because most creatures are alert to signs of danger all around, this suggests uncertainty as to an invisible creature not being heard, hence the Dexterity (Stealth) check as part of the Hide action. (The meaningful consequence for failure is spending an action for no gain in a circumstance where you effectively have limited actions.) If the uncertainty is removed, however, such as with a silence spell, then the creature is hidden, no roll, since they are not clearly seen nor heard.