Honestly, I might recommend something a little bit different. For example...
Swashbuckler 3/Beguiler 2/Fighter 2 (Beguiler is from the PHB II, so, if that isn't available, doesn't really work... perhaps Wizard, or Sorcerer, if you have a good Charisma).
Beguiler adds a few nice tricks - in combat, the best one is the addition of Mage Armor, which helps compensate for your lack of actual armor. Basically, as far as AC goes, it means you always have the equivalent of a chain shirt, and Canny Defense provides the enhancement bonus. The rest of their list is also fun - things like Color Spray for dealing with mooks, expeditious retreat for zipping around the battlefield, charm person, disguise self, and comprehend languages for non-combat... you get quite a few benefits from the first level spells. It may be worth it to pick up Practiced Spellcaster, so your spells last longer (a standard mage armor from you will last 2 hours; with Practiced Spellcaster, it lasts 6). Beguiler also gets you extra skill points if you take it at level one, and lets you claim any treasure that's designed for arcane casters (say a Ring of Wizardry), if, as you say, there are no other spellcasters in the party.
You might end up looking at something like this:
1. Beguiler 1. Practiced Spellcaster (Beguiler), Dodge
2. Swashbuckler 1. Weapon Finesse (B)
3. Beguiler 2. Mobility.
4. Fighter 1. Weapon Focus (Rapier)
5. Fighter 2. Combat Expertise
6. Swashbuckler 2. Improved Disarm.
7. Swashbuckler 3.
You can swap around when you take particular classes and feats, obviously, and you only *need* Dodge and Mobility from that list. Also, while it delays your entry into Duelist, you could drop levels of Fighter for more levels of Beguiler, if you want the higher level spell slots. A Beguiler 4/Swashbuckler 3/Fighter 1 gets 2nd level spells; a Beguiler 6/Swashbuckler 3 gets third level spells. The first adds things like Invisibility, Blur, mirror image, and spider climb, while the second adds displacement and haste, among others.
And after typing all that up, I'm now tempted to go that route myself...