I don't think it's contradictory at all - you might be invisible, but you still leave footprints, stir up dust, and disturb your surroundings in some minute way because of the air your displace.
A house rule that adds circumstance modifiers based on the surroundings would make sense - like a bonus to notice the presence of an invisible creature in a situation where their movements will definitely affect the enviroment (like walking on sand, through a very dusty room, or any surface that'll deform when someone steps on it like grass or a carpet), and a large penalty if they're walking through the middle of a spotless marble floor... But it definitely doesn't make any sense to rule that Spot can't possibly make you aware of an invisible creature, or replace it with a wisdom check - they're not ethereal, after all.
And there's nothing to stop an invisible creature from making Hide checks, to make the signs of their passing harder to spot.