But thanks to everyone for the Discussion. Ive still not quite made up my mind as of yet. I may even just take magical Initate (Warlock or Wizard) I also have even Debated at level 5 Dipping into Warlock. Has anyone done this? Anyone got a step by step build on what they had done?
My Warlock Swashbuckler was started at 6th level, so I didn't have to worry about building it much. I recommend Booming blade, as it is amazing for Bladelocks, if you land the hits. I wouldn't risk it on high AC targets, as multiple hits gives you a better chance to sneak attack, but lower enemies will have a lot of trouble. Booming Blade + Walk away will be awesome, and is actually what I will be doing every now and then for my guy since I just picked up BB myself. Next level I will be getting a second attack through the Invocation, so I will have even better melee damage on the turns I don't use Booming Blade.
As for a step-by-step build? I assume you are starting at 3rd level, otherwise you wouldn't already be a Swashbuckler. Personally, as soon as I got Swashbuckler I would be pushing Warlock. Pick the right spells and it can offset the lack of sneak attack damage, and waiting on the ASI. The Mage armor Invocation at 2nd level Warlock will be great, giving you a higher AC. If you want the Extra attack at 5th you will need to go Bladelock, which works great when you mix it with melee classes. You might consider your Patron, as there are some interesting choices. Fiend is really tempting, but not the only option by any means. If you are allowed to use UA, the total is 7 patrons. Pick the one that best fits your story in this case, or find the mechanics you like most and figure out a story for that patron.
Archfey gives an AoE frighten, allowing you to keep enemies off of you.
Great Old One is a little trickier, you will have to be creative to use the 1st level feature to your advantage. However, 6th level (if you go that far) will be great for a Rogue, though less so for a Swashbuckler who would be getting sneak attack anyway.
Fiend has all the classics, feeding on your fallen enemies, manipulating luck. A solid choice, if somewhat cliche.
Not much to Offer on Undying from SCAG, it has pretty focused benefits early on. Spare the Dying isn't bad though, if you find your friends going down often.
UA Undying
Light has a much nicer feel to it, nicer in the literal sense of not being Bad. Positive Plane of existence and all that. Good for a more Story driven character, and gets +charisma to certain spells so make sure you pick enough spells that fit for you to get the bonus. However, not much of a bonus for Melee types, except for getting that +Cha on attacks with Green Flame Blade ( I wonder if your DM would let you add your Charisma to the second target? Something you would need to bring up, if you go that route.)
UA Hexblade was made to be for Melee/Magic people, and it meshes pretty well with Melee classes. If you go this route, you can actually dump dexterity as a Swashbuckler since Hexblade gets +Cha to hits with one-handed weapons. You might talk to your DM about changing the Cursebringer from a Greatsword into a Silver Cutlass or Short sword, to better fit your character. Additionally, the crit on 19/20 against Cursed targets is pretty nice for a Rouge who does a lot of sneak attacks.
UA Raven Queen. Hmm, I never actually looked into this one. I don't remember there being much that would help a Swashbuckler though.
That is my 2cp. I don't have any experience with the other two pacts, so I can't help with what you should do if you decide to go Tome or Chain.