You've got to be doing something wrong, and its that you multiplied the first by 19/20 and not 18/20
Expression 1 - Expression 2 = -2/20 * (1d4+16+3d8) +2/20 (4+16+24+3d6)
= 2/20 * (4+16+24+3d6-1d4-16-3d8)
= .1 * (4+24+3d6-1d4-3d8)
= .1 * (4+24+10.5-2.5-13.5)
= .1 * (29)
= 2.9
So Dmg before - Dmg after must equal 2.9. No matter your hit rate(so long as you aren't missing on a 18 or above), dagger master extra crit gives you around 2.5-3 dmg per attack.
But compared to a rapier at 50% hit rate[dagger at 55] the dagger gains .05 x 2.5+16+13.5[over the dager master advantage above] and the rapier gains +.45 x 2 and .05 x 4
or the dagger master gains +2.9+1.6 and the rapier rogue gains 1.1 making the final difference, and i am going to stress this for a 1w attack before any other paragaon class bonuses, pretty much the best case scenario for a dagger master at about +3.4 dmg/attack over the rapier rogue.
For every extra [w] on an attack, the rapier gains .45 x 4.5 and .05 x 8 [=2.425 dmg/attack] while the daggermaster gains .4x2.5 and .15 x 4 [= 1.6 avg dmg/attack]. Making the net for every extra [w] an advantage for the rapier of 8.25.
Such, for a 50% hit rate rapier rogue your damage schedule in Paragon Teir with a +3 weapon looks like this
1[W]: -3.4 dmg/attack
2[W]: -2.575 dmg/attack
3[W]: -1.75 dmg/attack
4[W]: -.925 dmg/attack
5[W]: -.1 dmg/attack
before any other paragon class advantages
For every +1 of to hit above 50% hit rate, the rapier rogue gains .05x2 avg damage over the dagger per W. So if your average hit rate is 55% then your schedule looks like this
1[W]: -3.3 dmg/attack
2[W]: -2.375 dmg/attack
3[W]: -1.45 dmg/attack
4[W]: -.525 dmg/attack
5[W]: +.4 dmg/attack
If your hit rate is 60% then its like the below[before any paragon class advantages]
1[W]: -3.1 dmg/attack
2[W]: -2.175 dmg/attack
3[W]: -1.15 dmg/attack
4[W]: -.125 dmg/attack
5[W]: +.9 dmg/attack
If its 65% then you're looking at
1[W]: -3.0 dmg/attack
2[W]: -1.975 dmg/attack
3[W]: -.85 dmg/attack
4[W]: +.175 dmg/attack
5[W]: +.9 dmg/attack
And so on and so forth. Once you get to 75% hit rate you breach the advantage of a 3[W] attack but you're never going to quite breach a 1[W] or 2[W] until you get to epic tier and pick up light blade mastery.
Of course, all of these numbers are before any other paragon class advantages. A rapier wielding shadow assasin gets +1d6 sneak attack die against bloodied enemies or, assuming an enemy is bloodied say, 40% of the time you attack another .07 dmg/per 5% hit rate [and .12 for the crit 5%] adding another .75 avg dmg on to attacks at the 50% hit rate, .82 at 55, .89 at 60, .96 at 65%, 1.03 at 70% and 1.1 at 75%[so just add that onto the end of the schedules above]. And of course another 6 or so damage if the shadow assasin doesn't get it or is able to provoke OA's and not get hit.