The biggest problem is that the rest of the cast can basically sing circles around Lin. Oh, he's plenty good, we're not talking the same writing-to-vocals talent disparity as, say, a Bob Dylan, but he just doesn't quite hold his own among the rest of the incredible talent surrounding him.
An interesting tidbit of trivia, but Lin originally conceptualized himself playing Aaron Burr, and if memory serves didn't make the decision to cast himself as Hamilton instead until relatively late in the writing process. I personally think it was the right choice, but it's definitely one of the more intriguing casting "what-ifs", it's really hard to imagine him and Leslie Odom Jr switching roles.