Diminishing returns are fairly steep, unless you are imposing a lot of disadvantage on an easy roll or a lot of advantage on a hard roll. I see no balance issues with this--the main problem is that it bogs down play as everyone tries to stack up advantage dice. Here are the actual numbers:
If you need an 11 or better (50%) to hit:
The first advantage die is worth +5.
The second advantage die is worth +2.5.
The third advantage die is worth +1.25.
The first disadvantage die is worth -5.
The second disadvantage die is worth -2.5.
The third disadvantage die is worth -1.25.
If you need a 9 or better (60%) to hit:
The first advantage die is worth +4.8.
The second advantage die is worth +1.92.
The third advantage die is worth +0.768.
The first disadvantage die is worth -4.8.
The second disadvantage die is worth -2.88.
The third disadvantage die is worth -1.728.
If you need a 7 or better (70%) to hit:
The first advantage die is worth +4.2.
The second advantage die is worth +1.26.
The third advantage die is worth +0.378.
The first disadvantage die is worth -4.2.
The second disadvantage die is worth -2.94.
The third disadvantage die is worth -2.058.
If you need a 5 or better (80%) to hit:
The first advantage die is worth +3.2.
The second advantage die is worth +0.64.
The third advantage die is worth +0.128.
The first disadvantage die is worth -3.2.
The second disadvantage die is worth -2.56.
The third disadvantage die is worth -2.048.
Looked at a different way:
If you need an 11 or better (50%) to hit:
The first advantage die changes it to a 6.
The second advantage die changes it to a 3.
The third advantage die changes it to a 2.
If you need a 9 or better (60%) to hit:
The first advantage die changes it to a 4.
The second advantage die changes it to a 2.
The third advantage die changes it to a 2.
If you need a 7 or better (70%) to hit:
The first advantage die changes it to a 3.
The second advantage die changes it to a 2.
The third advantage die changes it to a 2.
If you need a 5 or better (80%) to hit:
The first advantage die changes it to a 2.
The second advantage die changes it to a 2.
The third advantage die changes it to a 2.
Since 95% of attacks in the game result in 50% to 80% chances to hit, all a second advantage does is typically drops the die needed from whatever it was, to a 2 or 3. The third advantage always (in the 50% to 80% chances to hit) causes it to go to a 2.
So, having a second or third advantage roll means that you need to basically roll a 2 on the dice. Why not just go with the rule: "You need to roll a 2 on the dice"?
I agree with you that it bogs down play for no reason.