But the other kind of consistency I'm after is over the range of "real world" animals. Animals are better represented than vermin. Animals have Intelligence scores (roughly) representative of their real-world faculties; vermin are mindless, like zombies.
Oddly, this is the area where I'm inclined to make a house rule because I don't agree.
I believe that real world animals have an upper end intelligence of around 5-6, and that some of them have a highly limited (by human standards) but real and useful innate capacity for language and culture.
The following creatures would certainly have above 2 intelligence in my games: felines, canines, dolphins, whales, elephants, monkeys, apes, racoons, parrots, crows, octopi and possibly also rats, pigs, bears, and ferrets.
Likewise, I wouldn't want to overstate the mental capabilities of a jumping spider, but as a creature capable of learning, memory, and modifying its own behavior it has an intelligence higher than zero. However, the vermin rules seem to allow more leeway for a Int 1 vermin than Animal rules allow for Int 3animals.
I should say that my desire for animals to have above 2 intelligence is not motivated by a desire for 'realism' alone and that in fact, 'realism' is not the most compelling reason here for having animals above 2 intelligence. By far the most compelling reason for intelligent animals is that the source material - myths, legends, fairy tales - all assume that animals are inherently intelligent and are if given cause capable of speach, complex reasoning, and so forth. The source material assumes a animistic world where the cause of every sort of motion of every sort is the violition of some intelligent being. That is to say the wind moves because it wills itself to move, and water flows down to the sea because it wills itself to do so. When you hold this world view it is natural to percieve animals as intelligent actors, and the stories that inform my setting have intelligent animal actors advancing the plot.
It's worth noting that the Mindless trait in D&D does not and never has represented a complete lack of intelligence. Rather it represents an alien and perhaps unfathomable intelligence which is so far from the way humans percieve the world that it cannot be compared to it.
As for how residents of the D&D universe would tell the two apart, all D&D universe vermin have Darkvision, whereas D&D universe animals have low-light vision. Several easily performed spells would also constitute good tests.
If snails are vermin and octopodes animals, what would the D&D conversion of its common ancestor (whether or not animals actually did evolve in the D&D world) be considered?
I find the question rather meaningless, in as much as it is deeply rooted in cladistics. If animals don't evolve, then the phrase 'common ancestor' is meaningless. If they don't evolve, then they need not have a common ancestor and if no common ancestor exists then it would be impossible and meaningless to describe one. Without evolution, every creature is potentially a unique and independent creation. Something created snails. Something created octopi. If there is no evolution at all, two snails of different species aren't even related to each other necessarily and don't necessarily share a common ancestor. It just happens that whatever creates creatures has a fondness for the snail concept and replicated it a couple of times. And, obviously, the creator has an inordinate fondness for beetles.
Besides which, I've already answered this question as it pertains to my campaign. The common ancestor of snails and octopodes is a tree. Of course, an student of the arcane could tell you that the Great World Ash is not a literal three dimensional tree since it is in fact infinite in size and multi-dimensioned and has no definite up or down, and that its apparant treeness is simply a product of that being the only way a mortal mind can cope with its reality. And they could also tell you that the tree is not the ancestor of snail and octopodes in anything like the same sense that its the ancestor of the Gods, in as much as the six divine families are literally begotten of the tree whereas snails and octopodes are manifested by tree in the same sort of way that for example a fire manifests light and heat.
it gets fuzzy, because unlike Aquatic, which is clearly defined as creatures living in the water...
I don't think that people in the D&D universe would define 'Aquatic' in this way at all. I think that rather they would define 'Aquatic' as, "Having a particular affinity for the element water." or perhaps, "Being animated by a water spirit."