The bard spell list isn't disjointed. It's inclusive. Bards don't get the entire list. The PC gets the spells learned and those only include spells you might think are not thematic only if you as the player choose to add those to your character. That falls into a self-fulfilling prophecy if that's how you feel about those spells.
Which leads to my next point: what you consider thematic for the bard doesn't necessarily mean another player feels the same way. A lot of spells in D&D are self-referential and don't draw on traditional mythology or folk-lore. Bard spells do, right up to power word kill.
The original bard spell progression from Strategic Review cast MU spells up to 7th level (the same spell level as most classes). Every wizard spell was a bard spell except for the highest levels. The 5e appendix optional bard was similar except it forced dual-classing first for the fighter and thief components before become a druid with extra powers.
Every druid spell was a bard spell except for the highest levels and druid spells were implemented to be somewhere between a magic-user and a cleric at the time. Then came the bard remade article from Dragon Magazine for players who wanted to play a bard but removed some of the thief flavor in favor of a stronger Celtic / Welsh inspiration (drawing from The Song of Rhiannon, for example). That optional bard could cast druid spells (the spell table went up to 7th level druid spells) and added illusionist spells up to 4th level on a second chart.
In all 3 cases bards were strong spellcasters. The difference in comparison to paladins and rangers is drastic. If we maintain that gap (which we saw in edition after edition) then moving bards down to more limited spell caster would imply moving other classes back down too if status quo is an objective.
2e bard just learned any wizard spell up to 6th level spells (again, other spell casters were limited to spell levels from 5th-7th level depending on spell casting ability score so the spell levels matched. This was the first official bard and could definitely learn spells on the list to which you refer to as anti-thematic. Bards used to have access to a lot of spells you consider anti-thematic.
I would argue the 5e spell list actually matches your idea of a bard better than those original lists. To answer your questions better (and shorter) those spells have to do with actual mythology and folklore and class roots in D&D from older editions.
Bards did not lose this in 3.x either. 3.x is the only version of the D&D bard that fell behind in spell levels because that's the first edition where wizards (formerly magic-users) weren't the only class that had access to 8th and 9th level spells. It's also not the full story because PrC's were what lead to 4e's paths and 5e's subclasses and popular PrC's such as the sublime chord allowed for learning other spells, and spells up to 9th level like other spell casters. 3.5 also used an awkward mechanic that gave different spells at different levels so bards had 8th level spells native to the bard spell list as lower level spells gained at similar levels. Plus the magical song abilities padded out spell power in addition to spells. The entire thing made for a decent spell caster similar to the 1e Dragon version or the 5e version that we have now.
4e used rituals for some of the anti-thematic spells you mentioned. I'll ask you this: what edition of D&D do you think bards could not learn and cast teleport? Because they've been able to teleport for 35 years now. ;-)
They gave bards magical secrets because:
- It was popular in 3.5 bard PrC's.
- It helped recreate some bard PC's from earlier editions where those spells were available.
- It enabled customizing the bard towards the character concept (ie thematics) the player was looking for.
- It facilitated the JOAT concept.
They are the JOAT, or renaissance class. They are primarily support and function in a role based on mythology, history, folk-lore, and legend. Classes were implement to fit archetypes, not roles.
Also, have you tried spamming bardic inspiration? Starting with 3/day (generally) isn't spammable. Getting up to 5/rest isn't spammable when spell slots are more numerous, skills are more resource free, and cantrips or rituals exist. There's enough bardic inspiration to make it useful, but it's still a limited resource.
That's like saying wizards are an incohesive niche because there are diviners, evokers, or illusionist and then there's also blade singers or war mages so there's an issue with the class. Subclasses are meant to help fill archetypes. Variety exists because there seems to be a lot more bard archetypes than you seem to be acknowledging. ;-)
Valor is based more on the classic skald or warrior bard.
Swords is based more on maneuverability and a swashbuckler.
Whispers is the shadowy manipulator.
Eloquence comes from the Greek philosopher poets.
Glamour is the musician.
Lore is the classic Celtic/Welsh concepts.
Archetypes, not roles.
The class doesn't decide that. The player does based on his/her/their idea of what his/her/their bard represents from whatever mythology, history, legends, or folk-lore on which he/she/they are basing the PC.
The classes and subclasses are tools to help players build a character. They are not there to force characters into someone else's idea of the character that person is going to play.
If I want a bard from Bard's Tale or a bard from Song of Rhiannon then I would like to have the tools to create either. I can be a buffing/debuffing musician story teller or I can be a magical warrior or I can be a magic-user calming the storms of druids during the Milesian invasion or I can be another concept the class covers.
5e's implementation allows for a lot of bard concepts. That's not different than any other class covering multiple concepts.
Again, only the one theme you have in your idea of the class and missing the part where most of the spells are not accessible. Players can only take so many spells and if a player cares about the theme then the player will select spells, skills, and a college to match that concept.
Because subclass features are just a class feature and they get a universal class feature at 10th level instead in the form of magical secrets. Magical secrets is designed to support the subclass choice.
That's not a valid point. Spells are sorted by spell level regardless of class. Iconic spells are a bit more sometimes but the general level is the same even if it's a bit of a level band instead of a flat metric. It's not different than access to an iconic spell like fireball, however. Having a slower spell access progression is irrelevant to how the spell level mechanic is applied, and there's more access in the form of domains than there is in magical secrets.
As I pointed out above, bards used to have huge spell list access. Magical secrets is imported from popular 3.5 PrC's directly into the class and helps recreate older edition bards. This is a bit of a different topic, but it's a good ability that tends to get over-rated. Adding a ranger or paladin spell doesn't have the same impact as it does when a ranger or paladin takes the spell because the bard doesn't have the other class features to support it any more than when a lore bard picks up fireball and still can't fireball as well as an evoker.
Magical secrets are also few in number and exist at higher levels. Taking 1 or 2 spells from the ranger list is only a very small portion of the list and not as significant as it might appear.
I think you have a clear idea on what you think a bard is. The issue I have with making a change to the class in a future edition is that the current implementation can let you play your concept and let me play mine, but changing the class let's you play your concept and forces me to give mine up to play your concept too. That doesn't seem like a move in the right direction for the class.
Another concern I would have is that the current implementation of the bard is a lot more popular than the 3.x version that you seem to want to recreate. It doesn't make sense to move from a popular implementation to a less popular implementation.
I would be more inclined to possibly break the archetypes up a bit. A person can make a fighter subclass and call it a herald or make a rogue subclass and call it a troubadour or make a second class and call it a minstrel or do all of that, but that's going to give up a strong bard concept that doesn't work well with the JOAT/Renaissance class concept as a subclass of another spellcaster so a full spellcasting bard is still going to be desirable anyway to fill that concept.
