I also remember reading they were invented to fight a player's vampire character called Sir Fang.
Yes.
So I wanted to do a short series on some of the history of D&D, and I thought I'd start with a fun and relatively easy one- the original class system, as codified in the AD&D (1e) Player's Handbook of 1978. For those of you unfamiliar with that 1e PHB, it was a...
www.enworld.org
Going back to seriously address the OP (
@Gradine ). I've been thinking about this for a while, as I don't recall ever seeing a definitive answer to this question. What I do want to stress (as someone who was familiar with the era) is that it just seemed
natural at the time.
I'm going to speculate a little, from a few different thoughts.
First, Gygaxian D&D (OD&D and 1e) wasn't about standardization. Things were created organically through the accretion of rules, and not through some kind of master attempt at balance with a lot of modern playtesting. So I don't think that anyone really sat down and was like, "Hey, some casters go to 7, and some go to 9."
That said, it is certainly true that Clerics (and Druids) were good in combat. Unlike the pathetically weak Magic User, Clerics (and to a lesser extent, Druids) were your second-line fighters. Their spells shouldn't be as powerful as Magic Users. In addition to just going to 7th level, they didn't have the utility or breadth or signature combat spells (fireball and lightning bolt and magic missile and sleep) that Magic Users had.
But to give you an idea of the difference- a MU had
78 spells to choose from between 1st and 3rd level, and
150 spells to choose from between 1st and sixth level. Clerics and Druids could choose between
36 spells between 1st and 3rd level and
66 spells through 6th level.
Looking back, though, the real question isn't about Magic Users ... it's about
Illusionists. Illusionists suffered under all the same combat disabilities as Magic Users, they were magic specialists, but they also were capped at 7th level. More importantly, they had
even less choice that did Clerics and Druids. Sure, they also had 36 spells between 1st and 3rd level, but they only
60 spells through 6th level. Heck, one of the seventh level Illusionist spells (one of the SIX they could choose from) was First Level Magic User spells ....

That's right, you could Alter Reality, or you could gain all the power of a first level MU.
I think that what we see is that there was a
de facto cap of 7th level for spellcasters that weren't MUs. This happened in Dragon Magazine as well for the NPC classes- the Incantrix, literally the "anti-MU," was capped at 7th level spells (and couldn't learn their 7th level spell until 17th level, the same time that a MU had 2 8th level spells).
This may also have been an artifact of the separated spell lists- one great advantage of having unique spells lists for each class that each spellcasting class truly felt different. One disadvantage is that ... well, making lots of spells is hard! If they wanted to create a brand-new spellcaster, they'd have to create a lot of brand-new spells. It was easier to just go to 7, I guess.
Like a lot of the early rules, I think that the unsatisfying explanation is ... that's just the way they were.