I agree that the disarm rules aren't great. I guess they are simple?
I think the +/- 4 size modifiers on combat maneuvers are bit much. Grapple, trip, and disarm are Strength based. According to the MM, the larger a creature gets, the stronger it is. A Large creature doesn't really have a +4 advantage to disarm, grapple, and trip, it has a +6 advantage.
Ideally, as a house rule I would implement a rule changing the combat maneuver modifier to "size AC/to hit modifier x (-2)", not (-4). In other words, Small creatures would get -2 to trip, disarm, and grapple checks while Large creatures would only get +2 (-1 x -2). Right now smaller creatures are severely disadvantaged when confronted with these special maneuvers, and I'm not sure how justifiable it is. (I doubt this change would get through committee, even when I start GMing again in a few months.)