I agree with the above posters that you should be able to get the type of feel that you are looking for by using either a monk or a rogue. If you don't want the monk mystical weirdness then you could go with either the Martial Artist, or the Maneuver master. The martial artist archetype even lets you qualify for fighter feats using your monk level starting at 4th.
Also, the Savage barbarian seems to fit the idea that you are trying to get, but you wont get fighter feats.
Basically if you want to play a fighter, they are really good at being fighters and you should do that. Armor is one of their things. There are however other options for playing characters without armor, and you should be willing to explore those.
What's more important? Playing the concept that you are going for? Someone that kicks ass without armor? Or playing a fighter so you can get weapon spec?