How did this work out for you?
Okay, so a year plus later, Bobby Joe Boudreaux is now a Fighter 1 / Totem Warrior 7. He's a lot of fun to play (dumb elf with an Alabama accent and a mullet), but so far he's not much of a "hitter".
Current stats are: 13 / 18 / 16 / 8 / 13 / 8. He wields a +1 short sword and a +2 shield, and also found an Ioun Stone of Protection, so he's nigh unhittable already with an AC of 22. Even better if the cleric throws Shield of Faith on him! He did not take the bear totem (shock!), but rather the Eagle (both times) so far. His movement is 45, and with a bonus action to dash he can get just about anywhere on the battlefield. This is honestly one of the most fun aspects, because when I pull out one of these abilities, everyone at the table goes, "Huh?"
My next level up will be barbarian 8. I was originally planning to take the Shield Master feat, but realized that the bonus to saving throws only applies if the effect targets only me. That seems like something that will probably never come up. The mini-evasion could be pretty helpful, though; and depending on which version of Jeremy Crawford's Sage Advice you choose to follow, the bonus action to shove with the shield may or may not be able to grant me advantage on any of my attacks.
So I am considering taking Resilient instead. I'm proficient in Strength and Constitution saves right now, and have advantage on most Dex saves (besides having a +4 already), so I could take Resilient (Wisdom) instead of DEX. I will probably end up just adding two to my Dexterity and moving along, though.