I'd have a look at Fantasy Grounds II. It can be run with many different systems and from what I gather adding in your own home brew campaign/rules etc is possible as well, although it may require buying the complete version.
However if I understand right, as long as the DM has the Complete version then the other players can use unliscened versions to play with the DM. Basically, everyone could chip in and pay around the same as if everyone bought a liscenced version.
Anyway, the 3d dice rolling is beautiful.
Definitely worth having a look at. I'm still mastering the program (only 2 sessions played so far) but I think it's fantastic. Once I have mastered it, it will be magnificent!