Well, since armor check penalties from multiple sources on the same character (ie. from chainmail, a shield, and carrying a heavy load) all stack, that sounds reasonable. Simply applying the penalties from all armor being worn by the composite entity of the mount and rider.