Yes. However, since they're usually Touch spells (unless it's something weird like Ice Knife), they'll still generally be pretty good at hitting foes at higher levels. (For example, when fighting a dragon with a 10 touch AC, a -4 isn't going to matter to a 10th-level caster.)