If it takes them five hits to beat an adult human into unconsciousness, then that outcome is significantly less ridiculous than allowing them to instantly kill an armored soldier in one hit.
If you want to argue that toddler-sized humanoids with sticks are outside the purview of the game rules, or that they couldn't reasonably kill an adult human regardless, then the setting details disagree with you. Kobolds and goblins exist in most published settings, and are considered to be a notable (potentially lethal) threat to non-adventuring NPCs.
Chimpanzees are significantly smaller than adult humans, yet they are mind-blowingly stronger than a human. Just because something is small doesn't mean it can't be strong. If it has the capacity to effectively use deadly weapons (which act as a force multiplier) then it is even more dangerous. A goblin or kobold does not necessarily have the same muscle strength as a human relative to their size.
However, I was clearly talking about a toddler with a stick. You could substitute any other creature that couldn't reasonably kill an adult human under normal circumstances even if it tried (a mouse for example). Suggesting that it only takes 4-5 hits from such a creature to kill the average human adult is clearly an unreasonable (silly/absurd) outcome. Allowing such a creature to deal even 1 damage is arguably not very realistic. At a minimum, they shouldn't be able to take someone to 0 HP.
Whereas someone with a lethal weapon can most certainly kill an armored soldier with a single attack. That's happened countless times throughout history. It's not an unreasonable outcome at all.
If the soldier had 8 HP instead of 1, and the PC fighting him had an 8 damage minimum (roll of 1 + 5 attribute mod + 2 sneak attack dice) then there's no discernable difference between whether he's a minion or just a low HP creature. He dies from one hit either way. You're suggesting that it's somehow problematic if he was a minion in this hypothetical, despite the outcomes being identical?
I mean, sure, if you let the PCs raise an army of toddlers with sticks and run around whacking soldiers for lots of combined damage, you might have a problem. I would, however, disagree that the problem there is with the minion rules.
I'm not trying to convince anyone that they should use minions if they make the game less fun for them. Obviously, if they aren't fun for you, don't use them.