Well, for one thing, you've had to make a host of changes, which is different from playing D&D as written. Many of the differences are pretty substantial in scope, and have some pretty dubious grounding. Where, for example, do elves in LotR really cast anything resembling a D&D type spell? You can make a limited argument that Elrond cast some sort of divination in The Hobbit, and may have used a healing spell of some sort on Frodo in LotR, and there is the control over the waters of Rivendell's river, but that's it, and those don't fit any known D&D spells with respect to their use (and at least two seem more to be simply "really good skill" rather than "spells"). Galadirel scried using her font, but that seems to have been a property of the font, not a D&D style spell. And you're "everything else goes by the book" leaves a lot undefined. Do ranger's get animal companions? Do paladins exist? Do they have special mounts? Is Aragorn really a plain fighter, or is he some other class? Does he cast spells?
But the real difference is in magic availability. Using the standards set in the various editions of D&D, magic items are just much more common, and much more prominent in D&D than they ever are in LotR. You have to radically alter how magic really works to make D&D work like LotR, unless you are willing to handwave away a lot. And that's why D&D is a poor fit for LotR - the volume of handwaving and customized rules starts to overwhelm the system. At that point, there are a lot of systems that work much, much better for getting a LotR feel into a game session.