I'm going to upset a lot of people, I can tell already based on the responses.
I think it is quite a big deal when there's a large age gap. Especially when you are younger, the size of that gap tends to be smaller than when you're older.
Before I go on, I know someone will say "I know so-and-so" and yada yada yada. I'm gonna speak in generalities, and that's anecdotal anyway. Also, I am well aware that maturity has many different aspects, and that not everyone progresses at the same rate. Again, I'm gonna use generalities.
First off, it's not wrong for a nineteen year old to think that someone sixteen is hot, or to be attracted to someone of that age. After all, you were only that age three years ago, and it does take a while to grow out of that. And I seriously think they're putting something in the water nowadays, because I don't remember some of these sixteen year olds looking that hot when I was in school. I really don't think I need to go into the potential problems of a relationship between a nineteen year old and a sixteen year old, other than to say that I personally feel it is very much a bad idea.
There is a vast difference in both intellectual and emotional maturity between a sixteen year old, a nineteen year old, a twenty-five year old, a thirty-five year old, and a sixty year old. Part of this is because usually people of each age group are focusing on different things. A large part of it however is that there just isn't the degree of development and life experience between them. That makes a relationship between age groups rather difficult. Not impossible, as everyone's anecdotal evidence shows. Just difficult.
Also, it's bull that age differences don't matter, because they do. Appearance matters. You may not care about it, but other people do. There's a very definite stigma that occurs when you're an older guy dating a younger woman, especially when it's nineteen to sixteen. You could be the nicest guy in the world, but many people will think that you're taking advantage of the girl, and hold that against you. Not the least of which would be the girl's parents.
Between the societal pressure to date someone around your own age (though lessened between 30-50, I think), the general differences in maturity, and the differing life goals between age groups, it's just not a good idea, in my opinion.
As for anecdotal evidence... my brother's friend is 16, dating a 20 year old. Most people think he's the scum of the earth, and his friend is quite mature for her age. One of my good friends, 24, dated a guy who was 33, and while it wasn't as big a deal on the societal front, he never really fit in with any of her friends, and eventually that didn't work out. I dated a girl in college who was a freshman, 19, and I was 21, and it didn't work out well at all... she ended up cheating on me then breaking up with me, because she wanted to see what else was out there. I on the other hand started making the shift towards putting a life together. And the cute girl at the radio station got really mad at myself and the pop director because we told her she was hot, we wouldn't date her because she was 17... it was the moral of the thing, we agreed it would just be wrong.