What does the tribal raider see when he looks at civilization? We tend to assume that he, like city-dweller, sees himself as "primitive" and civilization as "advanced," and longs to live in a stone building in a city, be part of the development of new technologies, and only doesn't because the city-dwellers are keeping him out.
But what we often see is that the raider's view of the world isn't "primitive" vs "advanced," but "strong" vs "weak." He sees agrarian civilization in terms of those sleepy villages of weak men, who waste all their time growing tired in the fields, who never learn how to wield spear or bow, who are fit for little more than to die by the sword when the snows melt and the time is ripe for plunder.
The history of civilization is hardly one of continual expanse. There are plenty of examples of those raiding nomads getting the upper hand over cities and showing very little interest in preserving or adopting their ways. To us in the 20th century, we've got that 20/20 hindsight that civilization ultimately won, but to a Comanche warrior in 1760, or a Mongolian horse archer in 1240, his territory has been continually expanding, he's slaughtered his enemies, and if the world of cities and farms has anything he can use to become a more deadly raider, he's had little difficulty obtaining it. Why would he ever adopt their ways? They are weak, and he is strong.