Looking things over, I'd take Fighter the rest of the way, ending at Barbarian 8 / Fighter 12. Certainly if you're taking Fighter any significant ways, you have to get to level 11 for the second tier of Extra Attack. Most significant gains are in the 7-11 level range.
You lose out on the +1 rage damage from 9th level Barbarian, as well as the critical hit die, but Fighter 12 gets its 4th ASI/feat. The ASI is unlikely to compete with the extra damage (assuming you've already gotten to 20 Str, and possibly 20 Con, by then), but the feat very well could, or at least provide utility that outweighs that. (If you're not using feats in your game, you may be better off getting the extra rage/crit damage.)
Battle Master and Cavalier both seem like they'd mesh well with the Barbarian heritage.
The main losses of not taking Barbarian all the way is that your proficiency bonus caps at +4 (so your to-hit and skill checks will always be 2 points short of what they 'could' be), the bonus critical damage dice (up to 3), the increased number of rages (up to unlimited at 20th, but 6 at 19, compared to 4 at 8), that your rages don't end, and the uncapped Str/Con (where both could potentially hit 24).
As far as rages are concerned, the main benefit is the damage resistance you gain from the Bear totem. With only 4 rages per day, and the potential of losing the rage early in some circumstances, you'll have to be a bit tactical with its usage. But with the benefits you can gain from Fighter, I really don't see it as any great loss.
If it makes sense for the character to switch classes at this point, go ahead and do so, and don't look back. This isn't a bad point in time to make that change.