Since it's not a weapon, a disarm attempt vs. a holy symbol would use the Strike Object rules, right? (weilder's dex bonus, +5 to ac for being in hand)
How have you played this out? Often the symbol is attached to a chain or something, and the goal is not to destroy, just to unhand...