In the light most favorable to the original statement, let's start the "late 70s" as being 1977, when Traveller was introduced, and the "early 80s" to be 1986.
Or you could ask what was meant. By "early 80s" I was thinking 1980 to (say) 1982 or thereabouts.
I really don't think you should be asserting your anecdote as fact ("In the late 70s/early 80s, the second-biggest RPG after D&D was Traveller.") because you definitely can't source it, and I do not believe it is true.
OK. I do believe it is true, at least for the UK I'm relying on representation in gaming magazines, and reports in those magazines from leading figures in the hobby at that time. It's not a claim grounded in scientific historical evidence, though.
As you know, it's hard to overstate how big D&D was back then. It's why "D&D" was a synonym for roleplaying in general for most people. If you treat it separately, it would take up the top four spots (OD&D, AD&D, B/X, DIY variations, in some order). If you treat it as a monolith, then it's going to be "D&D, then other stuff."
I am not distinguishing different forms of D&D. The fact that it is "D&D, then other stuff" dosen't seem to bear on the question of whether or not there is a second-most popular RPG.
I can also state definitively that no one I know played either Traveller because no place carried it; instead, Gamma World was our go-to. Doesn't mean Traveller didn't exist. But as a general rule, I wouldn't say, "Gamma World was the second-biggest game from the 70s to the early 80s" because that would be nothing more than unsupported anecdotal assertion, especially given that there are no reliable sales figures from back then (which is a continuing source of annoyance for some of us!).
Suppose that Gamma World was a popular game, after D&D: that seems to contradict the earlier assertion, to which I responded with reference to Traveller, that
one thing we often ask is why (for example) "Sci-Fi" TTRPGs aren't very popular.
If you go too far away from what you know, you can't "just play" because your brain is always trying to translate. For example, if the world doesn't have normal rules for physics, or heat, or operates like Tenet, then it becomes increasingly hard to just play.
On the other hand, if the world is too accurate (a realistic depiction, or a ... I hate to use the word ... simulation of an actual period) then you have a similar problem with immersion. Instead of focusing on the play, you will be focused on "getting it right." It's really the same issue as the world being too alien.
Ars Magic is set in our earth, but I've never heard of an Ars Magica game being derailed because of a focus on "getting it right".
Likewise for CoC.
Two examples that have been given are 7th Sea and Tekumel. As I already posted, I think 7th Sea's world is pointless; and Tekumel relies on tropes and ideas that are not familiar.
@Aldarc's comments about attitudes towards children etc are (in my view) correct, but I don't think this is about
familiarity. It is about
distaste. I suspect for similar reasons there is little appetite for RPGs that invite players to sincerely occupy the role of (say) white supremacists or petty drug dealers, except perhaps for niche games that take an ironic approach to these features of contemporary society (eg Kill Puppies for Satan).