Referring to the Bladesinger subclass option specifically I've run a lot of numbers in consideration for my next character build and I honestly feel like the choice is not poor at all from a damage or survival standpoint (granted, you will have less hp than a fighter or Barbarian). Overall I've been extremely impressed with 5th editions ability to keep damage pretty normalized over time if you build for optimum damage output across different class choices.
The way I personally plan on approaching the Bladesinger:
Race: Eladrin (as per the DMG, mostly for flavor, though I consider Misty Step 1/short rest a pretty major boon).
Class: Pure wizard 1-x (we start at 1st)
Stats: DEX 16, INT 16
Cantrips at 1st: Green Flame Blade, Mage Hand, Minor Illusion (debating thundering blade or sword burst, but not entirely positive there).
Spells at 1st: 1 - Absorb Elements, Find Familiar, Ice Knife, Mage Armor, Shield, Sleep
Basic approach 1: (My numbers assume hits, I haven't applied the sliding scale math for misses and advantage)
Melee single target - Dual wield short swords (2d6+3, 5-15, avg DPR 10)
Ranged single target - Long bow (1d8+3, 4-11, avg DPR 7.5) <Note: long bow remains better than Firebolt>
Basic approach 2:
Melee single target - Dual wield short swords (2d6+3, 5-15, avg DPR 10)
Melee multi-target - Rapier, Green Flame Blade (1d8+6, 8-14, avg DPR 10.5) <Note: familiar can push advantage on GFB>
Ranged single target - Long bow (1d8+3, 4-11, avg DPR 7.5)
Basic approach at 3:
Cast and maintain Flaming Sphere (2nd level) (2d6 fire, 2-12, avg DPR 7) <Note: This is when wasting your bonus action on TWF loses strength>
Melee single target - Rapier, Green Flame Blade, bonus action to ram with sphere (1d8+3+2d6, 6-23, avg DPR: 14.5)
Melee multi-target - Rapier, Green Flame Blade, bonus action to ram with sphere (1d8+6+2d6, 9-26, avg DPR: 17.5)
Ranged single target - Long bow (1d8+3, 4-11, avg DPR 7.5)
4th level feat choice: Elemental Affinity (fire) Increases DPR by [.166 (per d6) | .125 (per d8) | .1 (per d10)] of fire damage and ignores fire resistance (1s count as 2s on dice) or possible War Caster
Thank you TwoSix for correcting my math. I was over-valuing Elemental Affinity. I think its still super viable especially given the frequency of fire resistance, but it really doesn't change the average damage curve in a major way. I still think it feels good regarding minimums.
At 5th:
Either cast and maintain Melf's Minute Meteors or Flaming Sphere (3rd level) depending on situation.
Melee single target - Rapier, Green Flame Blade, bonus action to ram with sphere (2d8+3+3d6, 8-37, avg DPR: 22.50) normal
Melee single target - Rapier, Green Flame Blade, bonus action to ram with sphere (2d8+3+3d6, 11-37, avg DPR: 23.125) Affinity
Melee multi-target - Rapier, Green Flame Blade, bonus action to ram with sphere (3d8+6+3d6, 12-48, avg DPR: 30.00) normal
Melee multi-target - Rapier, Green Flame Blade, bonus action to ram with sphere (3d8+6+3d6, 17-48, avg DPR: 30.755) Affinity
Ranged single target - Long bow (1d8+3, 4-11, avg DPR 7.5) <Note: You kinda suck at range at 5 without burning spell slots, this is the one level FB would be stronger>
At 6th: (I'm going to toss some numbers here just for comparison)
Dual wield short swords (3 attacks): (3d6+6, 9-24, avg DPR: 16.50)
Dual wield rapiers (req feat, 3 attacks): (3d8+6, 9-30, avg DPR: 19.5)
Melee single target - Rapier, Green Flame Blade, bonus action to ram with sphere (2d8+3+3d6, 8-37, avg DPR: 22.5) normal
Melee single target - Rapier, Green Flame Blade, bonus action to ram with sphere (2d8+3+3d6, 11-37, avg DPR: 23.125) Affinity
Melee multi-target - Rapier, Green Flame Blade, bonus action to ram with sphere (3d8+6+3d6, 12-48, avg DPR: 30.00) normal
Melee multi-target - Rapier, Green Flame Blade, bonus action to ram with sphere (3d8+6+3d6, 17-48, avg DPR: 30.755) Affinity
Ranged single target - Long bow (2d8+6, 8-22, avg DPR 15.0)
Ranged single target - Fire Bolt (2d10, 2-20, avg DPR 11.0)
Ranged single target - Fire Bolt (2d10, 4-20, avg DPR 11.20) <note: with elemental adept (fire)>
I think the biggest concern with bladesinger is getting target fixated on that extra attack at 6th. I feel like there's absolutely no value in that feature unless you're in the super niche situation of spellcaster assassination/lockdown (i.e. drop a silence and now you outclass the enemy caster, or globe of invuln, or something similar). At no point is that extra attack stronger than GFB used at the same level (unless of course you end up with a +3 sword or something, these assumptions are for low magic campaigns).
While dipping into paladin does give you higher spike potential via Divine Smite, I hate that you can toss all of your resources in a round or two. Fighter is similar with Action surge, which is absolutely awesome 99% of the time.
Personally I prefer the idea of dumping (1d8+3[weapon])+(1d8+3d6[fire]) fire every round single target, scaling more multi-target for up to 10 rounds of combat. Shield and absorb elements should give you the staying power you need. Form an unholy pact with your cleric (if you have one) and get Warding bond up for even more fun. You always have the option of tossing a mean fireball or Melf's Minute if that's in your interest.
Note: I would personally suggest dumping counterspells like crazy as well, but my group's DM has politely asked that I not focus 100% on shutting down enemy spellcasters.
You also have an AC of 16 at 1 (mage armor), 18/19 at 2 (studded or mage armor), scaling with gear (+2 bracers armor, +2 if you get warding bond, etc) with the spike potential of +5 from shield for a round.
I personally don't see any weakness in damage output, or survival really except for the lack of HP. But if you take a big hit just misty step/ddoor and get the eff out of dodge. Fighters and paladins don't have that option. Hell, later on you can contigency ddoor or teleport if you don't like getting beat up. I just don't believe the build is weak, and I don't think dips are worth it overall until at best level 7 or 8, but I wouldn't give up spell and feat progression for a very minor gain in potential spike. IMO that's giving up 10d6 of flaming sphere damage over 10 rounds.
I think much more offensive bonuses built into the subclass would actually be OP as heck. And that DPR really isn't terrible, especially for a character that can still fireball, mirror image, teleport, and ddoor on their normal schedule.
Note: I also removed my chart because its just not accurate ATM.