If you use Mage Armor you're already better than all light armor unless you find a very nice magic one (which isn't guaranteed).
Getting high AC with a bladesinger is trivial. You'd want to start with 16 DEX and 16 INT (if elf, consider 17 DEX and taking resilient (dexterity) for the sex save). While bladesinging in mage armor that puts you at 19 AC. Having a shield spell handy pushes that up to 24 in a pinch.
At lower levels you'd still need to be careful, as due to your low HP you can't take too many hits. At mid-levels you get access to all of the amazing buff spells wizards have: mirror image, blur, haste, etc. These greatly help out with your survivability. You don't need resilient (CON) as you get the concentration buff while blade singing, though it wouldn't hurt if you have room.
At higher levels, after you've maxed out your INT and DEX, you'll have 23 AC with mage armor. 28 with shield. You'd also likely have mirror image or blur or haste up, further buffing your defenses. So while you're 'squishy' you're also very hard to hit.
The racial restriction is pretty much a DM call (so says the book), so if your DM allows it a variant human starting as a figher might be fun - for dual wielding. You'd start as a fighter for 1 level and grab dual wielder as your 1st level feat. At 4th level you'd need to grab war caster. Then you max INT, then you can max DEX after that. Dual wielding with bladesong is fun, allowed via RAW, and encouraged via RIA (take a look at the styles sidebar - a dual wielding handaxe style is mentioned, for instance).
The idea here is the +INT to damage you get at level 14. At max INT and DEX, and assuming a rapier, you're going to be dealing 1d8+10 damage per attack, and with dual wielding you get that nice extra attack. With three attacks, assuming all hit, thats 3d8+30 damage (averages out 43.5 damage).
The Dual Wielder feat also has the side-bonus of giving you more AC, which you really need to bolster your defenses. This build will almost never fail a concentration save: fighter gives it proficiency in concentration saves + bladesinging gives + INT to saves + warcaster gives advantage. Assuming a 14 CON and eventual 20 INT (and eventual +6 proficiency bonus), you're looking at +13 to concentration saves, with advantage.
The drawback of dual wielding is that it will cause troubles if you need a material components, but the assumption there is that you'll avoid such spells while in melee, and when being a 'normal' wizard you'll sheathe one of your blades so you have a free hand for components/a focus.
Starting as fighter also gives a side-bonus: skills. You can grab the Acrobatics skill (in flavor and something that will actually be very valuable to your fighting style), and even the perception skill. Take the sage background for knowledge skills or the urchin background for sneaky skills, and use your human proficiency and whatever you're missing (arcana or stealth, I'd imagine).
I would think you'd want a starting array of (after racial bonuses): 8 STR, 16 DEX, 14 CON, 16 INT, 10 WIS, 8 CHA
If you're restricted to elf, then you'd still want to use mage armor and a rapier ... however you don't really have enough feats to dual-wield, unless you're OK with skipping the dual-wielder feat or warcaster. I don't see dual-wielding really being 'worth it' unless you completely maximize it though. In this case you'd want to probably just go straight wizard: 8 STR, 16-17 DEX, 14 CON, 16 INT, 8-10 WIS, 8 CHA. Take the higher DEX/lower WIS if you want to grab Resilient (DEX) at some point. Otherwise max INT, then max DEX. You'd have a free feat slot which you could use for whatever, or you could grab two levels of fighter for action surge and the one-hand style (for a nice +2 damage). If grabbing the fighter levels it might be better to start as fighter for the CON save and Acrobatics (as well as a few extra HP, of course).
Overall, bladesinger looks to be trying to be the 'agile' frontliner. Very difficult to hit, but to make up for that very low HP.
