The bolded is where we disagree, I think. And if a player wants to have their character spend the time, money, and creativity to make the Broom more useful, I'm fine with it!
Note however that adding a saddle is going to make carrying passengers somewhat more difficult...
Then again, I also...