Personally, I might run this as instead of not adding the modifier, the damage die o the Bonus Action attack goes down one step (to a minimum of 1d4), and remains the same if you have Two Weapon Fighting. I would also just combine Two Weapon Fighting and Dual Wielder. More succinct this way imo.