I don't see how ruling that cloud giants can be seen automatically after breaking cover in broad daylight should be deemed a whim of the DM. You admit that it is ridiculous to rule otherwise and therefore common sense, not a whim, that this should not be the case. A huge chunk of the game is already at the whim of the DM from setting DCs to deciding monster hp.
Those same cloud giants doing that same thing under cover of darkness in a human settlement on the other hand - they would just have to beat the passive perception of the guards - likely a roll of 7 or more would suffice since humans cannot see in the dark (-5) so the cloud giants win as long as they stay out of torchlight. If the town is alert to giant attacks then the guards would be rolling perception checks and readying their interact with object to sound an alarm bell.