I can't believe that I can't find it... but the RAW for flying ranges is that you take the largest number between horizontal distance and vertical distance. So if the creature is 5 over and 4 up, then it is 5 squares away when counting range for attacks.
Edit: Ha! DMG 45, Movement in Three Dimensions, p5-6.