I'm going to weigh in with a previous poster about being aware of the consequences of the PCs' actions. If slavery in your campaign setting is a lot like antebellum slavery, then the PCs killing the slavers may not be a good idea. Let's be real here. Chances are the slavers and slave-owrners would believe the slaves responsible. They (I think) have the numerical advantage. They also (I think) know their way around the area, so sneaking in and out of a building could be very easy for them. And even simple farm tools can be used as weapons if the need arises. Sickles, picks, and pitchforks can work pretty well if everybody is asleep or otherwise not paying attention. Given how slave revolts can and do happen, the slavers would not be remiss in assuming (at first) that the slaves are responsible. Unless they have zone of truth and similar spells, they'd have no way of being absolutely certain that the slaves did not commit mutiny. So, even if the PCs do succeed in killing the slavers and freeing some of the slaves, I doubt the slave-holders would believe Diamondback responsible, especially considering that suspecting another slave-holder would make sense only if the slaves turn up on somebody else's land. Even if they did believe Diamondback did it, they may assume that the slaves cooperated for some reason or another. In any case, regardless of what is believed, it is very likely that violent action against slave-holders would result in the slaves being blamed and harmed for it.
As a previous poster said, slavery is messy.