The tricky thing is that a lot of Tolkien magic, at least on behalf of the "good guys", isn't spell-casting but simply knowing the right words. It is implied many times that the elvish langauge(s) can have what we would consider a magical effect (speaking to animals, awakening trees etc.) but it is more a facet that the elvish languages have a property that is closer to the language of the creator(s). It's a sort of pre-Babel speech of you like.
The elves don't consider many of their own creations "magic" (lembas, elven cloaks, hithlain rope, boats that don't capsize) - to them they are merely well crafted, perhaps "perfectly crafted" might be one way of putting it.
Magic in the form of spells and weaving of arcane powers seems more to be in the realm of the bad guys - the Witch-King is considered a potent sorcerer (heck, his lair is known as the "City of Evil Magic" - Minas Morgul). I think that Tolkien probably means such knowledge to be dangerous, and those who pursue it too much risk falling (it is, after all, pursuit of power which is a big Tolkien no-no). Saruman and Celebrimbor, for example, fall too mcuh to the "magic and artifice" path. Sauron the Crafty certainly does and his Valar kin Aule skirts pretty close at times.
It's notable that Gandalf tends to shy away from any overt magical displays, and many of his fireball type displays probably come from Narya as much as his own ability. Throughout the siege of Minas Tirith his power is more in "firing men's courage" than in blasting Mumakil with fireballs!