The moment you start assuming ANYTHING about the motives of the Patron or familiar, you've moved into speculation territory. The spell conjures a spirit, yes. Binds, yes. Obeys, yes.It's not speculation. Chain pact warlocks gain familiars via the Find Familiar spell, which conjures a spirit and binds it to your service. The spell even says that it always obeys your commands. I doubt most warlock patrons would subject themselves to magical slavery.
Anything else? You're guessing. The moment you said "I doubt" is when you went off track there. You're attributing motives and reactions that you feel is appropriate, but others disagree.
While, as DM, you can make any NPC feel any way, but there's nothing saying that a DM has to run a familiar/patron in the way you are suggesting. If anything, I'd say that's deliberate because it gets in the way of stories that you could run. Further, there's nothing to say that the familiar bond can't be broken by the familiar in the case of abuse (and, in the case of the psuedodragon, explicitly says that's exactly what happens), which makes it far less of a slavery issue and more of an "employment" one. If it was slavery, having a bound patron as a slave forced to teach you magic is very much in par with a warlock design as well.
We're talking about non-humans here, as well. The imp especially is part of an infernal hierarchy where obeying and pleasing superiors is an art form, and being on the material plane to gather souls or further an agenda could be germane to its ascension to a higher form. Psuedodragon is very much not human in thought or motive; I imagine its closer to an intelligent canine, who wouldn't have any issues with following orders of their leader. Sprites, likewise, could see you as an extension of a fey noble, so obeying the fey noble's representative is natural in stories of the fey world. Quasit? Demons are renowned for being balls of hate and destruction already, so that's par for the course, really.
So much potential for various stories here to limit it to "magical, unwilling slavery."