My understanding is that elves and fairies being small, tiny sprite like things is to some extent a product of the Victorians. Though in older tales there are at least two major categories of elves, some of whom are tall and other small, as mentioned, for example, here, by the folklorist Morgan Daimler.
I think there's an argument to be made that one of most well known examples of elves in popular culture (as opposed to folklore) isn't from anything we would consider (classical) fantasy or European literature, but rather from Twas the Night Before Christmas. It was an immensely important piece of American literature that helped shape the landscape of holidays in America and across the world (also coming to fame around the same time as another very important piece of Christmas literature: A Christmas Carol). And it names what may be the most famous elf of all time: Santa Claus.
In the 1800s many depictions of Santa were much more traditionally elf-like. Smaller, pointy ears, etc. That had a big change in the 1930s when Coca-Cola gave Santa his red suit and new visual aesthetic. Their advertisements effectively gave the world the traditional red-suit Santa that we know today. But he would still be considered an elf by many holiday and pop culture traditions. A tall, powerful, immortal elf with magical powers.
I don't really know where I'm going with this. Obviously, Tolkien's reaction to C.S Lewis putting Santa in his books is well documented, so I doubt he would approve of this comparison. But it would be super entertaining for someone to do a deep dive comparison of Elrond to Santa.