You still primarily fight in-melee but you do need to understand the context of the fight and adjust accordingly.
Regardless of the fantasy you want to buy into the monk, or even the intended fantasy, the truth is that their features funnel you into a playstyle where you're quick to engage with enemies that don't want to be in melee with you, and dispatch them quickly.
The fact that you are talking about the intended fantasy being at odds with the reality of their features? That thing? That's the thing we've been talking about as a bad thing. If the reality at the table is the make is better off not entering into melee with a melee combatant, then the reality for the designers is they messed up.
If you're fighting a bugbear and two goblins, the fighter/paladin/barbarian would focus the bugbear and force them to either take strong opportunity attacks and possibly outright die, or engage with the tank as they're supposed to. The monk, on the other hand, bothers the goblins by forcing them into melee when they prefer to be in range.
And I think its a nice fantasy to be the guy chasing down and dismantling 3-5 mooks in a fight while the rest focus down "The Big Guy".
Fundamentally, any warrior class should be able to chase down the mooks and dismantle them. Monks don't blow the other classes out of the water with their damage. They have better mobility than some, sure, but a Barbarian can easily reach 45 to ft of movement too.
The bad part of the design is that the Barbarian or the Fighter or the Paladin just needs a movement solution, something many of them have. They can switch between running down the mooks or focusing on the big guy. The Monk CAN'T. Fundamentally, they lack the ability to consistently fight in melee with the big guy. Yes, you can decide to play an archer instead of a monk, but there are better chassises to use to build an archer than a monk base class.
And frankly, I think the answer is just a little bit of a change. Just a few small things will fix the monk, and it won't even ruin this idea of them being the kite instead of the wall.
level 1:
- Give them a d10 HD
- Give them + 5 ft of movement.
- Give them Fancy Footwork
Level 2:
- Change the ki to be lv+wisdom mod
And that really solves their low-level problems. They are still mobile, kiting, warriors who don't stay in melee, but now they don't need to spend ki every turn to retreat after getting in melee. If they are in melee, the d10 HD gives them just a bit more survivability. It doesn't seem like a lot, but since your typical monk is going to have a +1 con that raises them from 9 hp to 11 hp. Two hp isn't much, but it means they can survive being hit twice by a goblin, where before they couldn't.
And then you increase the Ki, so they can use their abilities a little more often. With the Fancy Footwork ability (which most monks spend their level 4 feat to get because it is so vital) they don't need ki to disengage, unless they are dealing with a large number of foes, and they can flurry, disengage, and move.
They still need a damage boost at level 11, but I'm serious, I think those small changes at levels 1 and 2 flip the switch and make all the difference in allowing the monk to stay mobile and not get bogged down in combat, use their abilities more often, and feel better able to survive on the frontline if it becomes necessary for them.