I voted history/background. But that really isn't it. Making an item intersting involves a combination of factors in my mind.
First off, a magic item needs a name to be interesting. As long as a magic item is called a +2 Flaming Longsword, it will never transcend its stats. Once the sword is given a name, such as "Pyre" or "Flametongue" or "The Torch of Mars" or something like that, it begins to have an identity. From there, it is a matter of building on that identity, until the magic item is memorable and interesting.
In my opinion, a good magic item is an NPC into itself, even if it isn't intelligent. Like a memorable NPC, a good magic item may have a back-story, memorable personality quirks, distinctive features, and so on. An interesting physical description for the weapon helps a lot. A magic sword that quivers if it is covered in blood for too long is also interesting.
Also like an NPC, a magic item can be made more memorable and interesting depending on how it is introduced and how it works into the story. A magic item that is given to a PC as a symbol of the princess's undying love has more sentimental value to a player than the same item found in a dragon hoard. The famous magic sword that can only be wielded by someone of a certain bloodline (which serves as proof that a PC is a member of that bloodline) has an indispensable story role.
In summation, it takes a similar kind of effort to make a +2 Longsword into an interesting and memorable item as it does making a Level 5 Lawful Neutral Human Fighter into an interesting and memorable character. As an alternative analogy, making a magic item interesting is like making a dungeon or city memorable or interesting.
I guess that my final answer is that a magic item needs personality, charm, and character in order to be memorable.