I assume you haven't actually tried fighting with different weapons at all seriously.
My bonafides: At various times in my life, I've tinkered around with unarmed martial arts, I was trained as an Army Infantryman, though I rode a desk, and for the past seven years I've made a fairly serious study of classical fencing-foil, epee, and saber, and Western Martial Arts, specifically the rapier, cut and thrust and longsword. I'm really good with Hutton saber, and am working on Polish/Eastern European saber.
If you look up "salamandyr" on youtube, I've got a couple videos up of myself and of few of my fellows bouting, if you want to check them out.
So that's what I've got going for me. You said that you are studying katana; good man. Keep at it, the benefits of swordsmanship are grand!
The thought that the olympic gold medalist archer would last more than one round in a club-level martial arts is, frankly, a little silly. Even if you restrict just to ranged weapons, a high-school javelin champ will beat the ranger. Even if you restrict to a really focused category, like weaponless martial arts, I guarantee that a decent club-level brazilian jiu-jitsu fighter will take apart the TKD world champion at his sport.
OK, here's the problem...these are all
sports. We're talking about combat. A fighter isn't someone who studies one thing, archery, say, to the exclusion of all other things, so that he becomes the best there is at using that particular thing. A fighter doesn't study archery at all...or fencing...or jiu-jitsu. A fighter
studies how to kill people. The bow, the sword, the bare hands, those are just means to an end, not the ends themselves. So, like the special forces soldier, he or she relentlessly cross trains, because he or she never knows what will be available when the need arises.
This is a traditional area of debate in game systems, the "natural talent" versus "training" debate. In the real world, you absolutely need training to be vaguely competitive. And weapon type is such a factor their are bazillions of rules in competition to make sure the playing field is level
There are no rules to combat save for "Don't die!" and "Pointy end goes in the other person!".
I have had 2 years training with a katana. Let me totally assure you that if, in an empty room I started 40' from anybody in the world with a nail file, the chances of them winning a fight with me are very low. And I'm now a barely fit 46 year old. I could even tell them my strategy, which is to assume the standard hold and advance on them and as soon as they are in range, diagonal downward cut aiming at the neck/shoulder join. Since they have zero parrying ability, their choices are basically to strike first or dodge. Both require them to be able to close 6' faster than I can bring down the sword, which is possible, but really unlikely. I have fought (in TKD) the US lightweight junior champion who was ungodly fast and even so, I'm pretty sure I'd cut him before he got to me.
In real life, most swordfights ended with one person dead and the other person injured--true. And in general, if you are unarmed (or have a nail file), you will lose against a moderately trained person with a sword.
However, part of learning swodsmanship learning how to deal with a swordsman while unarmed. Sure your chances are less, but they're not
zero. Let's take you, a moderately trained middle aged swordsman, and put you up against, not a lightweight junior champion, but a Navy SEAL, or Delta Force soldier. He's unarmed, and he's not really a champion of any particular fighting style. He's probably never faced a swordsman in his life, armed or unarmed. However, odds are he is going to take that razor away from you and feed it to you, because he's a warrior. That's what warriors do. If he took an injury he it would probably be a cut on his off-hand, sacrificed to immobilze your blade while his dominant hand dealt with you.
And that's dealing with the difference, in game terms, betweens someone 1st or 2nd level, and someone about 6th level. That's not even discussing the worlshattering might of a 20th level fighter.
Fighters being even close to equally good with weapons they have never used before, or even weapons they do not regularly train with, is total fantasy -- which is fine -- but it's not vaguely realistic.
I'm going under the assumption that by the time a fighter gets to the level that weapon choice becomes almost irrelevant to him, 10th level and up, there really doesn't exist a "weapon he's never used before". He's either used it, or he's used something so close that the moves translate. Or he's just so fundamentally familiar with body mechanics that he almost instinctively knows how and where to attack.
Basically what this comes down to, is that somebody said something to the effect above that "people should be scared of a barbarian wielding a giant axe". I agree, I just think what scares them should be the barbarian, not the axe.