Turns out Tavern Brawler doesn't even need to be adjusted for it to work with a longsword.
"In many cases, an improvised weapon is similar
to an actual weapon and can be treated as such. For
example, a table leg is akin to a club. At the DM’s option,
a character proficient with a weapon can use...