You say that penalties last til after your next extended rest. What about conditions? That would be extremely harsh and would grind the adventuring day to a standstill if you got, say, a 1. Personally I would change it to "Save Ends" on those conditions.
BTW, I like rituals too, but have...