It's not long at all. You just don't let the dragon turtle because that doesn't work and you know it.
Just to be clear what we are talking about here:
I'm talking about having the dragon stay in the cloud until its breath weapon recharges, exiting the cloud to attack, and then putting up the cloud of darkness again.
On average, the dragon will spend two-thirds of the encounter in the cloud, during which time the melee attackers will be waiting on the sidelines and the ranged attackers will be mostly missing. However, the ranged attackers are likely to be still attacking (and taking up time), and the melee attackers may also be switching to their weaker ranged attacks (and taking up time).
If the PCs bunch up before the dragon attacks, the dragon uses its breath weapon and tries to catch as many PCs as possible. If the PCs spread out before the dragon attacks, the dragon focuses on the one it assesses is most vulnerable and either attacks with double attack (if it can reach him with a move action) or a charging bite (otherwise), and then when the melee attackers close in, uses his breath weapon to catch as many as possible.
When the cloud of darkness is not up, the black dragon isn't any less dangerous than any other 4th-level encounter, e.g. an encounter with five kobold slyblades - it has more hit points, its defenses are on average four points higher, and while it can't deal concentrated damage, it can deal mass damage with its breath weapon.
While the cloud of darkness is up, even though the dragon won't be able to attack a clever party, the average party's ability to damage the dragon is also minimal. So, the cloud of darkness effectively drags out the fight and buys time for the dragon's most powerful ability (its breath) to recharge.
Given all of the above, do you still believe that:
1. A young black dragon presents approximately the same level of challenge to the average 4th-level party as any other 4th-level encounter.
2. The average 4th-level party will be able to defeat a young black dragon without using more resources than they would normally need to use against any other 4th-level encounter.
3. A fight against a young black dragon will not be significantly longer or more frustrating for the average 4th-level party compared to any other 4th-level encounter.