RogueJK
It's not "Rouge"... That's makeup.
Taking levels of anything before you get to Bladesinger 6 for extra attack will weaken you by dealying an already delayed "5th" level power bump. As well as 3rd level spells like Haste. I could see starting with one level of fighter for CON saves, but anything more wait until later.
You can lean on Booming Blade/Green Flame Blade to help bridge the gap left by the delayed Extra Attack. Same as any other gish build that doesn't get Extra Attack, like an Arcane Trickster Rogue, a Paladin2/SorcererX, a Hexblade Tomelock, a melee Arcana Cleric, etc. BB/GFB's extra damage scales with character level, not class level, and helps keep your melee damage in line overall with Extra Attack progression. (Albeit a bit more "swingy", since you're relying on just one attack to either hit for a bunch or miss altogether, so it's all or nothing).
However, I'll concede that the delay to 3rd level spells like Haste is a bit more troublesome. You have to ensure that the 1-4ish level dip is worth the cost of delaying your higher level spells, or at least have another dedicated buffer in the party. (Besides, it's better for a melee combatant to be Hasted than to Haste themselves, as it lessens the chance of losing Concentration.)
I've played an Arcane Trickster/Bladesinger from Level 3 (2/1) through Level 10 (3/7) to good effect, without feeling delayed/underpowered. Booming Blade made a noticeable difference, and synergizes well with the Rogue's Bonus Action Disengage too.
So provided you're dipping into something directly useful to your Bladesinger side, like Fighter 1, Rogue 3, or Artificer 3, the delay can still be worthwhile.