So you want to get rid of Divine magic....
Send your party on some dungeon adventure.
Have them get trapped in statis for a few hundred years. Or accidently time travel. Or shifted to an alternate version of the world. Etc.
Whatever the cause, when they get out, they'll discover that the world isn't quite the same as when they went into the dungeon.....
For starters, those channeling the divine are cut off. They can believe & pray & preach all they like, but there's no power being granted.
Why? What happened to the gods?
? Maybe your pantheons version of Ragnarok happened?
? Maybe the gods just finally lost interest in this world & moved on?
? Maybe some other faction took over/drove them out? You say your OK with druids. So what if the druidic patrons, fey, etc finally "won", forcing the other divine pantheon(s) out once and for all.