Instead of going in the front, go up and to the left when outside, and you'll find a bunch of drunk, sleeping hobgoblins and goblins. Be careful and don't wake them (so don't like, jump near them), and you'll find a damaged, breakable wall. To smash that wall takes bludgeoning damage and it's loud so will wake the goblins if you're not careful. You can either talk them down, or use a Silence spell on the area to avoid the sound (the game is fairly well-coded about understanding Silence, I note).
You are then inside the temple, but up on the rafters, which are literally 30ft up, and you can basically avoid the vast majority of the temple if you want, and even steal most of the good loot so long as you have a good jumper (high STR, so Lae'zel or Karlach) who can jump down and shove stuff into their pockets or send to camp. You will need a box or two at one point if you have any STR 8 characters to jump up, but a high STR character can easily find some about the place.
I went this way and did the Dror Ragzlin fight first. I had Lae'zel and Karlach pick up some of the smokepowder barrels from a back room I found, and we stood in the rafters above him, and just hurled them down on him - I didn't manage to stop the alarm - if you're smarter than me just destroy the drum first. Anyway extremely heavy barrels plus violent explosions as they hit naked flames = dead hobgoblin. Plus if throw say, a spear down 30ft/10m, you get a +13 to damage on top of other bonuses, and there's a large chance the enemy is prone'd. Goblins get up there, Karlach or Lae'zel can easily just click throw on them and send them back down at their friends, potentially killing two goblins with one throw. Other enemies can be punted off highly effectively too. You will experience some archers etc, but they have penalties (you're in the dark, up from them, and often out of range of their Darkvision), so throw weapons down at them and they get splattered.
I don't even know how the fight is supposed to go because I've only ever done it this way.
Once he's done, Minthara is nearby and relatively easy to deal with, I found. She used to get kicked into the lava or the bridge she crosses got destroyed but now she's smarter and uses Misty Step to get to you, generally avoiding that. So then you have the most challenging fight from this angle, with some kind of goblin caster in her little throne room thing (Priestess Gut?) and a bazillion allies. Best thing you can do is trigger the fight from the side of the spider pit distant to her (closer to Dror Ragzlin's room), and then make the goblins come to you. EIther that or if you have smokepowder left, sneak someone up to behind her with it, drop it off, and detonate it.
Make sure you don't go out the front door after that though, because oh my god that will not go well.