For a single dice it's
avg(exploding dice) = average(dX)/(1-P(max))
where average(dX) is the average result of the dice used, non-exploding, and P(max) is the probability of throwing the maximum.
So, for d6 it's 3,5/(5/6) = 4,2, d8 = 4,5/(7/8) = 5,1428 etc..