You just asked what is probably a very complex question. I don't claim to know, though I can toss around some ideas.
Affinity for anime seems to be directly correllated to relatively early exposure. For instance, my early exposure to Teenage Ninja Team Gatchaman (via Battle of the Planets) in the mid 70's gives me some appreciation for Anime even though I'm highly critical of it as an art form. People without the early exposure rarely seem to develop a taste for it. (I'm sure there are exceptions.)
Also, anime is underground, gothy, taboo, and well 'naughty'. It is therefore hip to like anime and wear anime related articles in order to firmly place yourself in a particular subculture. But in my experience, more people like anime than actually watch it. And more people say they like it than actually like it. Most people who do like it, have a small selection of anime that consider 'the good stuff', and tend to snear at those whose selection is different than there own. This may or may not be related to the nostalgia thing I just mentioned.
In general, anime is alot cooler to look at than watch. Anime is a very stylized, very visually evocative art form. Even bad anime can produce very interesting visuals.
A couple of people have mentioned the porn angle, and that is not to be underestimated it. Anime is generally pornagraphic or at least sexually stimulating, with highly exageratted idolized femine forms - very large prominated breasts, huge eyes, impossibly long and slender legs, narrow hips, long necks, masses of hair that seems to have a life of its own, and very revealing costumes that also seem at times to have lifes of thier own. What hasn't been mentioned is that on the other hand, in addition to these subservient female ideals, anime features alot of powerful heroines who if not capable of controlling thier sexual urges, or at least capable of holding thier own in combat and otherwise taking care of themselves. So, while not perfect, anime does have the angle of presenting young women with high central capable heroines that are generally lacking in movies.
Also, it is really important to put Anime in context. At the time say Gatchaman, Robotech, Voltron, and Speed Racer came out, they really were light years ahead of most serious american animation which had arguably peaked in the WB years back in 40's and 50's. They were new, they were different, and they were clearly targeted at a more mature audience than the alternative. So, in context, it is very easy to understand how anime got a stong underground following in the college age and younger crowd. As a preadolescent kid, those shows blew me away.
However, I do think that US anime has caught up to and even in many cased passed the Japaneese stuff if you compare good US toons to good Japaneese toons, and not lame US toons to good anime or big eyed, big headed, kids anime (the equivalent of say smurfs) to good US cartoons. It is hard to argue that shows like GI Joe, The Dungeons & Dragons Cartoon, early Transformers, Batman: The Animated Series, The Simpsons, Pole Position, Alf Tales, The Totally Misfit Misadventures of Ed Grimley, The Real Ghostbusters, and so forth are not as well written (or better) than their Japaneese counterparts. Likewise, it is hard to argue that the above shows don't obviously owe a great stylistic dept to earlier Japaneese anime.
The real problem is that anime is on the whole just lame. The writing tends to be terrible and seldom really addresses anything like a deep or original idea. There are occasionally nature stewartship messages, and consequences of violence messages, but any sort of thesis that might be presented is generally buried under the incoherent and illogical activities that occur for the sake of presenting strange visuals, scantily clad females, and stylized light flashing combats. In general, Western viewers - including myself - have long written off the incoherent plots as the inability to relate to the Japaneese mythos or problems in translation, but the more j-toons I watch the less I'm able to convince myself of this. People who are fluent in Japaneese may feel free to correct me if they like, but I really think that there isn't an excuse - the writting is usually bad, choppy, childish, vulgar, and the plots muddled and trite.
That isn't to say that the whole artform is bankrupt, just that there is less quality on the whole than there might be, which I something you could say about movies in general. There is certainly anime out there that isn't a waste of time, but in general 'anime' is not proof of superior quality of storytelling to say Jem and the Holograms or Rainbowbrite.
