They would fight. A shepherd would fight a wolf among its sheep, and so a pit fiend would fight a balor.
A shepherd would probably also eat meat from time to time, and what meat is available...
To OP: A party without levels where full casters get 8th levels spells should probably not stay long...