Trying one more time to work out the Vex math. Going to brute force it instead of relying on math tricks, which I seem to be screwing up.
Let's start from the first swing without advantage. How does Vex progress from that?
Base damage: 4.5 (1d8) + 4 (Dex) = 8.5 + 5% * 4.5 = 8.725
Adv damage: 4.5 (1d8) + 4 (Dex) = 8.5 + 9.75% * 4.5 = 8.939
EDIT: Argh. OK, the crit damage calculations aren't done correctly when the accuracy is applied in the table below, rather than in the equation above. However the difference is like 0.1 damage, and won't be relevant to the final answer, so I'm not going to mess with it further.
The table is the chance the previous hit landed times the chance that the hit landed with advantage (higher damage due to higher crit chance) plus the chance the previous hit missed times the chance that a baseline attack will land. Anything else is a miss and doesn't contribute.
Code:
0: ~ 60% * 8.725 + ~ 40% * 0 = 5.235
1: 60.0% * 84% * 8.939 + 40.0% * 60% * 8.725 = 6.599
2: 74.4% * 84% * 8.939 + 25.6% * 60% * 8.725 = 6.927
3: 77.9% * 84% * 8.939 + 22.1% * 60% * 8.725 = 7.006
4: 78.7% * 84% * 8.939 + 21.3% * 60% * 8.725 = 7.024
5: 78.9% * 84% * 8.939 + 21.1% * 60% * 8.725 = 7.024
6: 78.9% * 84% * 8.939 + 21.1% * 60% * 8.725 = 7.024
At this point the math is stable, where the chance to have advantage in the next round is the same as the previous round. It reaches a long-term average damage per hit of 7.02, compared to the base average of 5.235 per attack, which is a 34% gain if you can maintain that streak on a single mob.
On the other hand, if you kill a mob in, say, four hits, your overall damage boost is about 23% before it resets back to baseline.
Checking at 70% base accuracy.
Code:
0: ~ 70% * 8.725 + ~ 30% * 0 = 6.108
1: 70.0% * 91% * 8.939 + 30.0% * 70% * 8.725 = 7.526
2: 84.7% * 91% * 8.939 + 15.3% * 70% * 8.725 = 7.824
3: 87.8% * 91% * 8.939 + 12.2% * 70% * 8.725 = 7.887
4: 88.4% * 91% * 8.939 + 11.6% * 70% * 8.725 = 7.899
5: 88.6% * 91% * 8.939 + 11.4% * 70% * 8.725 = 7.903
6: 88.6% * 91% * 8.939 + 11.4% * 70% * 8.725 = 7.903
Again we've reached a stable point, with a Vex average damage of 7.9 compared to the baseline 6.1, for a 29% gain.
If you kill a mob in four hits and have to reset, your effective gain is about 20%.
If advantage is particularly useful to you (for example, a melee rogue wanting sneak attack on a mob he's fighting solo), it's obviously far better than the numbers suggest. If you just want it for better accuracy, and you can kill the mobs in a few hits, it's giving you about the same benefit as Graze (around 20%). For a longer fight against a boss mob, it will provide a better benefit than something like Graze.