Paul Farquhar
Legend
To clarify (since paraphrasing seems to have muddied):
Starting with a level of Fighter gets you;
- More health to start with.
- +4 to +6 to attack rolls (+2 from Archery fighting style, +proficiency from... being proficient).
- Proficiency in all armours.
Doing so effects you thus:
- You get your ability bump at level 5, not level 4.
- You get your second level spells at level 6, not level 5.
- You get your second attack at level 7, not level 6.
As you can see, the downside is far less than "screwed".
* 2 more hp won't make much difference at level 4.
* You get Martial Weapons proficiency anyway from being a valor bard, so that's +2, not +4 to +6
* You don't need heavy armor proficiency, since you have a high dex for archery. Shield prof is nice, but you can't use one at the same time as a bow.