Doesn't pose a huge problem for some characters, but for others. . .
Fighter might roll 1-3 die per attack, but a Sorcerer doing say, 3d6 to 5 different enemies, you have just sentenced him to a ton of rolls, and the DM to a ton of bookkeeping.
"Okay, roll damage die for the ones with one hit...