It should be noted that this would mostly be an issue at lower levels. At higher levels, your ability bonus to damage becomes a smaller and smaller part of the whole. At 1st level, a rogue doing just 2d6 damage with a sneak-attacking rapier without ability bonus to damage is kinda weaksauce, and getting a +4 to that would be a huge improvement. But if we forward to 7th level you'd be hitting for 4d6+2 (1d6 bases, +1d6 for Striking, +2d6 sneak attack, +2 weapon specialization), which makes the +4 a much smaller part of the whole. Move up to 16th level, and you're doing 6d6+6 before adding +5 (or +6 if you got an early apex item) for your ability score. And that's assuming you don't put any elemental runes on your weapon, which could be adding another d6 or two, and that you don't have any Strength bonus at all to damage.
Magical weapons have two types of runes on them: fundamental runes, which are what make the weapon as such better, and property runes which add additional abilities. Fundamental weapon runes come in two further categories: potency (+1, +2, or +3 to hit) and striking (1, 2, or 3 dice extra weapon damage – so a striking battleaxe would deal 2d8 damage). Property runes are basically everything else you can do with a magic weapon, e.g. ghost touch, returning (on a thrown weapon), and so on. Quite a lot of these add 1d6 energy damage of some sort (flaming, frost, shock etc.), and these are colloquially called elemental runes, and they make pretty good runes to default to – basically, if you don't have a special plan intended for your weapon (like returning on a throwing weapon), you won't go wrong with an elemental rune for some extra damage. You can have a maximum of one property rune per "plus", so a +2 weapon can have two property runes.
Potency/Striking runes are an essential upgrade for every character who intends to actually use a weapon. Without getting those at the levels where they become available, you will lag behind quite a bit when it comes to combat ability.
Oh, and bards generally don't make good melee combatants in PF2. Bards are full casters, with the weapon proficiencies to match, meaning they only become Expert in their weapons at level 11 and never become Masters in fighting. Even the bard subclass that's supposed to be good at combat (Warrior muse, from Advanced Player's Guide) only get broader weapon proficiency, not better. I mean, it's not wrong for a bard to have a weapon for use when they have actions to spare, but swording opponents as a bard is at best a plan D (with plan C being arrowing them instead)
If you do use weapon Bard stuff, expect it to be a third action after you've gotten most of your effectiveness out of a spell. To explain:
Naturally, a basic Martial has 2-3 points of accuracy over you (they're usually a proficiency rank above, and for some levels their starting 18 vs. your starting 16 gives them the extra point) and the most basic martial routine is to strike twice (once at full bonus, and once at -5, -4 if you use an agile weapon.) That second attack the martial treats as worth doing? Your first attack still has a higher accuracy bonus than it assuming the martial isn't doing something fancy, so one attack per turn is surely worth doing.
Of course your attacks are less effective because you don't have [sneak attack/rage/precise strikes/hunters edge/legendary prof] to boost your damage, but you're still a full spellcaster so the two actions you spent before or after that on your turn need to be recognized, your attack is the icing on the cake for the effectiveness of your turn, not the meat and potatoes.
Though, fun fact-- Bards are an interesting choice for this, because of Inspire Courage, you can Inspire Courage, likely with lingering composition, boosting your other martials for a few turns, move into position to provide flanking and be getting, effectively +3 on the strike you close out your turn with, while providing that same bonus to someone who can hit more reliably and harder, raising their hit and crit chance both by 15%. Then next turn, you have that +3 in place through your flank and lingering composition already in place and can say, cast a healing spell or a control spell, and still swing.