Yup, that's true. I was intending on doing this without the need for the tavern brawler feat simply by being liberal with the improvised weapon rules which already allow for the use of the proficiency bonus if the DM permits (that would be me and I do).
As Shiroiken points out, this would...