Great Old One
Can either one be created by a player using the PHB and-or house rules? (assume for the nonce that a player for some reason wants to play a merchant)
No, they can't. The merchant has expertise with no levels, and the bodyguard has abilities that no class has.
If yes, all is good. If no, there's a problem.
And which one is that ? They are both specialised, but not along the paths of the PCs. Is there some cosmic rule that prevents this ?
Also, were it me I'd stat out the bodyguard as if it was a PC anyway, because for all I know the party will take a shine to him and try to hire him into their crew!
What do you mean "hire" ? If it's as a henchman, where would the problem ? A player could even play it as a side-kick and it would be way easier because it's simple. And if a player want to replace his PC, why not, he is absolutely welcome to a character that is absolutely unable to progress further as it has reached the limits of what his abilities and choice of training can give him.
Because that's what PC represent, exceptional characters with almost unlimited advancement, whereas most people in any world (and if you want verisimilitude and "realism", this is exactly what you should have) cannot progress past certain points.
In any case, it's in part for reasons like this that I don't and never will use 3e-5e style feats. Bleah! For me that bodyguard would just be a lowish level Fighter specialized in unarmed combat and with an unusually high (but PC-achievable!) Wisdom or Charisma* for a Fighter.
And he would be cinematically poor and very uninteresting as an adversary, whereas my bodyguard will make a very intersting foe especially if the PCs are in a hurry to kill the merchant, and have either to go through or around the bodyguard. By doing things by YOUR rules, you are depriving yourself of very interesting possibilities.
You and others keep raising the spectre of a huge amount of work, and this is a straw man. There's nothing stopping you from just quickly choosing and assigning stats, feats, abilities etc. rather than rolling them - the only constraint is that the end result be something that could in theory be generated by rolling it up the long way.
And, after doing that for years in 3e especially at high level, it's not a strawman at all. It once took me three hours to create a NPC, and after that I just said "screw this I have better way to spend my time" and completely skilled the "check that the end result could in theory by generated by rolling" and the next one took me a few minutes and was just as satisfying to play. All the bonuses where totally winged and certainly incorrect, some abilities were certainly incorrect and not available, but he was as fun to play and as much as a challenge.
So sorry, no, it's not a spectre, it's absolute reality, and many people here tell you this from experience.
Now if 5e gives you too many feats and abilities etc. to choose from and thus bogs down the process that's a fault of the system.
3e was way worse, for 5e it's so easy to take an existing monster/NPC and just take another ability that looks fun in another stat block and just play that way. It certainly could not be generated (which a constraint that YOU impose on YOURSELF for no good reason other than your personal view that it should be available to player), but it's blindingly fast (much faster than generating a character using the PC way), and it creates many more possibilities which are way more consistent with the setting. Why should an expert merchant always be an outstanding thief ? Where is the setting consistency in that ?