Druids have to be somewhat neutral in order to have proper attunement to nature. Nature is beyond morality. Druids must exercise some degree of indifference or balance in themselves if they are to have a suitable affinity with nature. Just as clerics need to conform at least somewhat to their deity's principles, a druid must share at least some of nature's indifference towards morality, or sense of balance via cycles, in order to tap into nature's power.
Druids do not have to be all the same in mindset, and don't need to serve all sides in the alignment clash. Balance is not their prime concern. Nature is. Any one side of the alignment axes gaining superiority would almost certainly result in the exploitation, ruination, or disruption of nature, and that is the only reason druids give a nod to universal balance. Law would harvest and exploit nature for maximum persistant gain, heedless of disrupting natural habitats, and would seek domination of nature to build cities and machines and other unnatural junk. Chaos would just destroy random chunks of nature as individuals did whatever was best for themselves or their families. Good would 'tame' and convert nature to make farms, cities, fruit groves, and so on, and would try to beautify unsightly parts of nature, to the detriment of marsh and desert ecologies for instance. Evil would destroy, despoil, or exploit nature for fun and profit. So druids would rather see the conflict of alignments remain in relative balance so as not to let such forces focus their attentions on harvesting nature. As long as nature is everyone's ally and nobody's friend, it can likely maintain its cycles in relative peace.
Druids do not have to be immoral or indifferent, but they do have to share some of nature's philosophy at least. Druids are individuals too, just like paladins, and bards, and rogues. Some will truly revere nature and serve it, others will just respect and use it, and still others will think they revere and serve it even though they really just revere a certain part of it or a certain view of it.
True neutral druids would be the sort that care most for nature's philosophy, having the strongest likelyhood of being indifferent, immoral, balanced, and extreme in mindset; they would be the preservers of balance, ensuring that nature survives by keeping the alignment conflict from resolution, and siding with whoever will best serve nature's survival and health at any given time, but generally ignoring the activities of civilization and alignments except where it impacts nature; they probably don't care too much about the rise, fall, lifestyle, laws, and day-to-day actions of civilizations, but will deal with them and intercede on nature's behalf when it seems necessary or important for nature's continued well-being, and only insofar as it serves that purpose; they probably don't care about the construction of homes and cities and such, so long as it isn't too rapid or too disruptive to nature's ecologies and resources, but they can be pretty vengeful and terrifying when they do decide that people have exceeded the reasonable tolerances of nature's serenity; they won't often interfere in worldly affairs, or they might only interfere when, where, and to-whatever-extent is necessary to maintain nature's health, or when it is important to the survival and health of fey, plant creatures, animals, and such; or the true neutral druid might only care about nature's preservation and purity, regardless of all else, doing anything to ensure nature's delicate balance and healthy cycles, even if it means reducing all intelligent races to living like primitives just so they can't muck up nature anymore with their infernal cities and machines and wars and farms; they won't want everyone dead, but they may want everyone to stop using up more of nature and stop ruining it, so they may do whatever seems necessary to put a halt to civilization's growth and advancement, such as inducing plagues or crop blights in areas to stem their growth and maybe set them back for a while.
Neutral good druids may be compassionate, but still understand the need for predators and survival of the fittest; they will serve nature's kinder, more beatific side, and seek amicable relations between civilization and nature; they will work to prevent excessive exploitation or conversion of nature, but they won't oppose natural exploitation in moderation or by sustainable ways; they might be smiters of those who exploit nature and those who carelessly destroy peoples or nature; people are part of nature too and deserve to live, even if it's alright for some to die as far as nature is concerned, but genocide of a race or civilization is excessive.
Neutral evil druids may be vindictive and predatory, but still see genocide of races or civilizations as excessive in most cases, and know that predators need plenty of prey to survive for future generations to feed on; they would not allow the extinction of useful or worthy prey, nor of important or useful predators, but they don't mind the destruction of particularly weak or useless races; they would not allow anyone to get too close to destroying the world or its life, so even neutral evil druids would oppose any attempts at world destruction or domination; they might test creatures or civilizations to see if they are worthy of life, or to see if they deserve to be preyed upon for the greater benefit and health of nature; they might be amoral avengers of nature, destroying anything and everything that messes with nature beyond what is needed to survive in good health, so they wouldn't oppose every new house, but they would oppose any continual expansion of towns, trying to kill off enough of the population to maintain a more static town size; or neutral evil druids might simply be people who use nature for their own gain, but in such a way as to still serve nature in some way, since nature needs predators and a certain degree of destruction to clear away unwanted chaff or to allow for new growth and adaptation.
Kinda drawing a blank on chaotic neutral and lawful neutral druids at the moment, as they're somewhere close to true neutral druids in mindset, but also have a few tendencies similar to neutral evil or neutral good druids...... CN druids obviously use nature for themselves or simply idealize nature's primal, destructive, reckless, opportunistic, and free aspects, with a decided hatred of civilization (or at least big or complex civilization), preferring solo, clannish, or tribal lifestyles for everyone. LN druids instead revere nature's order, persistance, inexorability, constant cycles, renewal, and dominance, or might utilize nature for the benefit of society at large so long as that utilization can be sustained indefinitely, even if it is to the detriment or demise of some people or civilizations or creatures or freedoms.