When I imagine combat in my mind, I imagine two fighters swinging their swords like crazy at each other. Swords and fists and shields are clanging and sparking everywhere, six, seven, twelve times per round. It's a melee...literally.
Now for most combatants, there's a chance that one of these dozen-or-so swings will impact the target in a six-second period. If one of those combatants has the Extra Attack feature or a weapon in their off-hand, there's a chance that maybe two of these swings will impact the target. If one of them is a monk and spends some Ki points, there's a chance that four of those swings might hit impact the target. There's not a single "whiff", there are several...too many to count.
The idea that a trained martial artist can only swing once or twice every six seconds is immersion-breaking for me.