I think that one level of fighter is extremely beneficial with very little drawback. You get CON Save, A fighting Style and Second Wind. The drawback is your spells come one level later; however, you only lose one spell slot. You will benefit throughout the whole carrier of the character from the fighter level and only during one level for the spell slot, and its iffy if you will even reach that level.
Also, another question is whether maxing out at 18 rather than 20 is really that important. That's two points of damage and 2 points of AC which will already be very high. If you Max out at 18 you can have the following feat tree
4th - Two weapon fighting
8th - War Caster
12th - Toughness or Int if you feel Toughness is not needed. You can use spells like False Life to substitute for HP
16th - Int
19th - Dex