I disagree with this valuation. What you're saying here is that at least 8 times in 9 the creature you hit with an acid arrow is still standing a round later ... and the creature may still be standing sometimes, but not 8 times in 9. Even when it is, delaying that damage in some of those situations will allow the creature one more round of combat, and that is a huge cost - a rare occurrence, but when it happens it can be big.
Melf's can do 4d4 now and 2d4 at the end of the next turn if you make a spell attack roll with a 90 foot range (half 4d4 on a miss).
Scorching Ray does 3 ranges spell attacks for 2d6 each at 120 foot range. Miss is no damage, but unless you're extremely unlikely to hit, you end up with higher expected damage with scorching ray at all points in time.
I made a similar spell to Melf's for my game call Acid Javelin. Also second Level. One attack roll. It does 4d6 acid damage on a hit (half on a miss) and the target makes a dexterity saving throw at the end of each of their turns or takes 2d6 more acid damage (save ends). I consider that much stronger than acid arrow - and people still do not want it.
Above, my "super melf" does 5d4 on a hit 23d4 on a miss. Each turn it loses 1d4 (2d4 if immersed in water), and you can use an action and take 1d4 damage to reduce it by 2d4.
That ends up pretty strong. If we drop it to 4d4/2d4 instead, we get something that is only marginally better than scorching ray:
So 4d4, 3d4, 2d4, 1d4 on a hit, and 2d4, 1d4 on a miss.
With 30% discount on delayed damage a hit is worth 10 + 5.3 + 2.5 + 0.9 = 18.7 on a hit, and a miss is worth 5 + 1.8 = 6.8. Or 11.9 P + 6.8 damage.
Scorching Ray does 21 damage; it needs a 75% hit chance to match it.
MM2 is 14 hit or miss. So this outdamages MM starting at 60% hit rate in a level 2 slot.
At higher levels, adding 1d4 to the hit and 1d4 every 2 levels miss has non-linear returns (as it lasts longer). "off to infinity" that 1d4 would be worth 4.1P + 4.1, which is a bit faster than the 7*P damage of scorching ray or the 3.5 damage of MM, but the "off to infinity" approximation is overestimating the value by a fair bit (like 30% at level 3? Decreasing at higher levels.)
I did a bunch of experiments a while back comparing the value of "damage now" versus "damage later." There are a ton of variables, but the rough guideline I emerged with was to discount the value of damage by 11% for each round delayed.
By this guideline, Melf's acid arrow dealing 10 this round and 5 next round would be equivalent to a spell dealing 14.45 damage straight up.
I agree with Stalker0; I suspect something is off with your math.
Maybe you are using lots of big monsters that last a long time, with plenty of time to drop the damage-over-time early, and knowledge that the monster lasts a long time?
I suppose most tough fights, by your measure, end with a long series of at-will damage output, because to hit 50% death rate you either need enemies who deal a pile of damage (with high variance) really fast, or you need to run out of resources and be on a race to death.
...
If you want to do this seriously, you'd probably want to brew up an AI learning system. You'd have them fight a sequence of random toughness encounters; maybe you'd feed the AI the CR of the encounter, as PCs know that an ancient dragon is different than 3 goblins.
For a given loadout of a party, you'd let the AI optimize how long they last against such a random sequence.
Feed something like that to Alpha or similar learning AIs and you should get resource management falling out of it. I mean, they can feed it starcraft and it beats grandmasters.