The "leap" you describe is caused by the Ranged Attacks in Close Combat rule employing a fortune-in-the-middle technique. As the rule states, all that needs to be established before rolling the dice is that a creature has the described positioning relative to a ranged attacker. The hostile attitude of the creature is a meaningful part of that positioning because it means the creature will take risks, if necessary, to oppose the attacker's actions. With that intent in mind, the attack is rolled with disadvantage, and the result informs any fiction that is then created about what the creature did to make the attack more difficult. By creating that fiction first, before mechanical resolution, you're adding an additional rider (e.g. that the creature must also distract the attacker) that is not present in the rule itself.
This is an inference on your part, same as mine.
Let me try to lay out both our arguments in a premise/conclusion format.
Hriston Premise 1: The rules state that a hostile creature within 5' imposes disadvantage on ranged attacks.
Hriston Premise 2: There is no exception stated for Hidden foes.
Hriston Conclusion 1: A ranged attacker with a hidden foe within 5' suffers disadvantage.
Hriston Corollary 1: The designers intended for Hidden foes to interfere with shots in the exact same way as obvious threats.
Hirston Corollary 2: The designers intended this to be an example of Fortune-in-the-middle design, where the rule comes before the fiction, and we create fiction to match it.
Mannahnin Premise 1: The rules state that a hostile creature within 5' imposes disadvantage on ranged attacks.
Mannahnin Premise 2: This appears intended to represent the fiction that with an immediate threat in melee reach, a ranged attacker will be distracted defending themselves and unable to optimally aim and fire their ranged weapon.
Mannahnin Premise 3: If a Hidden creature makes its presence known/draws attention to itself, it loses the Hidden status.
Mannahnin Premise 4: Actively touching and interfering with the body or equipment of an enemy is consistently something which has to be declared as an action, and involves some sort of roll, in D&D.
Mannahnin Premise 5: Nearly all mechanics in D&D employ Fortune at the End; we establish the fiction, and the rules follow from and implement that.
Mannahnin Premise 6: 5E is deliberately written with simpler, less exhaustive language than the two prior WotC editions, intending to leverage player "common sense" to adjudicate corner cases and unusual situations.
Mannahnin Conclusion 1: While the rule for nearby hostile creatures imposing Disadvantage on ranged attacks states no exception for Hidden creatures, this situation does not appear to match the fictive justification for the penalty that I have shared understanding of with my various gaming groups since playing this edition. There is a conflict here between rules and fiction.
Mannahnin Corollary 1: The lack of an exception to the ranged attack rules for Hidden foes may be an oversight, or an example of WotC trying to keep the rule simple and not expecting this situation. The correct ruling is ambiguous.
The invisibility spell has as a theme through the editions of "if you mess with someone else you lose the invisibility". There have been many ways to frame and phrase this. This edition goes with the simple "The spell ends for a target that attacks or casts a spell." Since they are interacting with the archer, however gently pushing the bow, they are adversely affecting another individual. This is enough to disrupt the spell to me.
For the record, Invisibility has consistently, through the various editions since 1974, had "attacking or casting a (n offensive) spell" as the condition which ends it. Not merely interfering with or touching someone. I don't think there's real grounds in the rules for saying that doing so would end Invisibility. It's whether it would end the Hidden condition that's at issue.
