Seems rather simple. I think I get how it works, but I have some suggestions:
There's no reason not to take full ranks in Concentration every level. A character would be foolish not to, because if he did, he'd be much weaker than usual. Basically all you're doing is reducing the number of skill points a spellcaster can have available.
Instead, why not just make it a caster level check + Charisma? Reduce the DC slightly to compensate for the fact that most people will be rolling d20 + level + Charisma instead of d20 + level + Charisma + 3. Maybe DC 5 + spell level x 2.
As someone else pointed out, fatiguing a wizard is no big deal, since they don't often need to use Strength or Dexterity. Rather, I'd say you should have a failure cause a -1 penalty to Cha (in addition to having the spell fail), so that it becomes progressively harder to keep casting spells. The penalty goes away with 8 hours of sleep.
In your current way, casters will just keep casting until they pass out. Even if they're nearly unconscious, most will want to keep casting. They can't do much without magic, and so it's generally better to keep casting (which will be just as easy when exhausted as when they're fresh). If you revise it this way, casters will reach a point of diminishing returns.
Also, why make spells easier to cast out of combat? This basically makes non-combat spells (like scrying and buffs) 3 levels lower.