I think that using a lethal weapon to deal non-lethal damage is like improvising a weapon. The sword deals 1d8 damage because it can cut you. I don't think it should deal that amount of damage if you're trying to hit someone with the flat of your sword, which requires to be careful in the way you hit. I'd certainly favor a club over a sword if hitting with the sword edge was our of the question.
So I'd rule that you need to advise in advance if you're trying to knock out instead of killing. Then, unless you have a suitable weapon for this purpose, you use the improvised weapon rules. My understanding of these rules is that the weapon attack is made without the weapon proficiency and deals 1d4 default damage unless you as DM feel that the weapon should be more effective and deal more damage (as they mention, picking up a table leg might be equivalent to a club). Personally, I'd go with no proficiency bonus/1d4 damage for most stuff; and make a few exceptions, including the staff, the club, punches (e.g. monk's), and allow those to keep the proficiency bonus to attack rolls but reduce their damage to 1d4 (or, if you prefer, keep their damage, but remove the proficiency bonus to attack rolls.)
I would stick with this only.
If you want to have a chance of killing the opponent instead of knocking him out, in addition or instead of the above, you can introduce a death save mechanic for the opponent: you have the opponent make a single death save when he's knocked out, on a 10+ on d20 he survives, on a 1-9 he dies.
And maybe, on a 20 he becomes immediately conscious at 1 HP. Yeah, that would be fun

Thought you had him...
This is a simple rule, easy to implement and remember.