Is there any decent background on how the New Republic failed so spectacularly to prevent the rise of the First Order? One expects that after the death of the emporer the Empire was shattered, and the New Republic would have asserted itself as the dominant power and would have, over several decades, overcome the bastions of the Empire.
Yes sir. The background is quite extensive, found in the new novels and the books like the Visual Dictionaries and such.
The short version: The war came to a head, and the last battle was fought over Jakku. The Empire--what was left of it--retreated to the Unknown Lands after signing a strict peace treaty. The Empire was, by the treaty, no longer able to build weapons or wage war.
Still, just like in real life, there are sympathizers with the old ways. There are Liberals and Conservatives, though they're named differently. Some still believe in the Empire and think, with Palpatine and Vader gone, that the Empire is the best way to continue, politically, without the stink of evil and the Sith. A "good" Empire.
Of course, those who wanted a return to the old ways won out, and the New Republic was established. But, even amongst the New Republic, some believed (Leia) in a strong military. Remember, in the prequels, it was established that the Old Republic did not have a standing army. There was a small policing force, and there were the Jedi--the guardians of peace in the Galaxy. That's what Attack of the Clones was all about--Palpatine raising an Army for the Republic.
Others went against Leia and Ackbar, wanting to go back to how it was with the Old Republic. Mon Mothma was among these people. These people argued that the New Republic would never truly replace the Old unless the military was disbanded.
Mon Mothma's side won out. The large military built up by the Rebel Alliance was disbanded by 90%. Only a small, peace keeping force was retained.
In TFA, most of this peace-keeping force was destroyed when the capital system of Hosnian Prime was destroyed.
Part of the New Republic structure was that the capital would change between worlds from time to time. That's why the capital was no longer Coruscant. It was thought that the Empire had infected that place, and it would be better for the New Republic to start fresh with a new, rotating, capital.
Still, the New Republic had its Empire sympathizers--those who found it was politically incorrect to announce their views. Even New Republic senators secretly wanted a return to the Empire sans Palpatine and the Sith.
The First Order sprang out of the desire of these people (Palpatines New Order morphed into the First Order). Secretly, senators and influential people funneled funs to the First Order, growing out of the old Empire in the Unknown Lands.
For 30 years, the First Order grew, secretly constructing ships and building upon the old Empire.
Leia heeds the rumors she hears of the First Order (this is covered in the book, Bloodline) and goes to the New Republic to martial their forces in anticipation of this threat. But, not everyone feels the way Leia does, and the New Republic does not officially take a stand against the First Order.
Instead, some in the New Republic secretly aid Leia in forming a political faction, the Resistance Against The First Order. Leia contacts those she has known through the years from the Rebellion, and the Reistance is provided with older, Rebel surplus gear.
The Resistance is in the open and is a greatly inferior force than is the First Order. The First Order has been quite secretive, but recently, especially in TFA and TLJ (which takes place immediately after Episode VII), is beginning to reveal itself more and more--especially now that the New Republic capital and fleet had been destroyed by the super-weapon from Starkiller base.
Of course, at the end of TLJ, the Reistance is just a few dozen people and a handful of ships.