If it's any consolation, Einstein would have probably agreed with you. He didn't believe that singularities were possible, and even wrote a paper attempting to prove this. I don't know if he ever changed his mind about this before his death
Not publicly, he didn't. But he died in 1955, before many of the major bits of work on them were completed. The very term "black hole" is post-Einstein.
but there were a lot of aspects of quantum physics, including black holes, that he really didn't like.
Yes, well, the existence of black holes owes more to Einstein's own General Relativity than to quantum mechanics. In essence, all QM does is fail to provide forces sufficient to prevent gravitational collapse. The singularity that Einstein didn't like is in his own equations of General Relativity, not in QM.