I liked the idea of applying a condition rider to the spell, but that seems a little powerful so I'm dubious.
What if you stole the bit from Shocking Grasp about targets of the spell losing their reaction? It would make it extremely useful in certain niche situations.
Is it really that powerful? Let's take it being in a second level slot.
Obviously, round one is not that powerful. We have pages of posts here saying that the spell is totally lame.
Round two. The PC has been standing out in the partial open, 30 feet away from the foe. The PC did 13 points last round. In order to do 13 points this round, the PC loses his action. The PC cannot critical. The PC cannot change targets. The PC cannot have a different concentration spell up. The PC has few options to react to changing situations without dropping the spell. The monster or an ally only has to do 1 point of damage to the PC to force a concentration check. The monster only has to temporarily walk behind cover to stop the spell. The monster only has to get 35 feet away to stop the spell (which a simple disengage does).
So at best, the PC does 13 more damage (the party Rogue at level 3 fighting two weapon is often doing 10 to 13 more damage, both attacks have to miss to do zero damage). In the middle, the foe is disengaging and losing one action. Somewhat less worse, the foe or an ally damages the PC and the spell is lost. And worse case, everybody and his brother drops down an attack on the PC trying to disrupt the spell, the PC keeps making the concentration checks, but he takes a lot of damage and possibly falls.
The difference between other first level spells put into higher level slots and this one is that the PC is seriously hampered with this spell. He loses an action every single round with this spell and he cannot seek heavy cover. He cannot cast other concentration spells, in fact he cannot cast any other spells except bonus and reaction, and he makes himself a target.
Any time that a spell caster has to maintain a spell to this degree, there should be a decent benefit for doing so. IMO. D12 per round is not a decent benefit.
Btw, Burning Hands cast in a second level slot against 3 or 4 foes can do 18 to 52 points of damage in a single round. Modified Witch Bolt averages 13 points a round spread out over many rounds. It could potentially take 4 rounds to do as much damage with Witch Bolt as it takes to do with Burning Hands in a single round.
Let's look at 3rd level spells. Fireball for 28/14 points of damage to multiple foes in a single round and the PC is free to do whatever he wants the following round, or houseruled Witch Bolt 20 points of damage a round against a single foe for multiple rounds and the PC is hampered and a target. That really sounds fairly balanced to me. I suspect that the vast majority of encounters, using Fireball in the third level slot will do more overall damage than using modified Witch Bolt in the third level slot.
Granted, one has to look carefully at Warlock and Sorcerer abilities to ensure that there is no synergy issue there. For example, doing it at 60 feet range is safer than doing it at 30 feet. Course, we have no warlocks or sorcerers PCs in our group, so this only helps the DM (since she might start throwing NPC PC-like foes at us).