We had one out-of-castle confrontation in the church of Vallaki, where he kidnapped Ireena.
Even without his lair actions, I find that his legendary action to move his speed without provoking OA's is more than enough to let him get out of any melee he cares to not want to be in. I made use of
spiderclimb there, too. Charm hasn't been
too effective yet (got some lucky PC's.

). All that +
Misty Escape assures that he won't be killed before the PC's do their due diligence.
My players are smart, so they're lookin' for the treasures before they go confront him. Strahd is also not opposed to them at the moment - they are the thing that gave him Ireena, that enabled her to have hope of leaving, and that hope lead her to him, to be bound forever. He's invited them to the wedding (hehehehe).
When Strahd does get furious, I plan on having him use his Children of the Night and lair minions (wolves, bats, rats, specters, shadows) to disarm the party of these items and abscond with them before he even bothers to deign to fight them. He'll focus on short-term harassment when the party is at it's most vulnerable (resting, stumbling through traps, caught at a dead end), especially with that
charm, retreating if they bother to do any notable damage to him, and ensuring that they cannot rest.
The only place he will stand and fight for any length of time is the location described in the Tarokka reading, where he'll remain until he's mist-ified.
And then he'll just take a quick dirt nap and be back before too long, so when the party fights him there, they better have already found his
actual tomb. Which should give Strahd plenty of opportunities to harass them.
I imagine when the big fight happens, it'll be like a horror movie climax: short, but
vicious. It's not about wearing the target down over time, it's about suddenly seizing on a momentary advantage before the oppressive evil simply resumes it's menace. The party's going to have to be remorseless and persistent.