I'm 52 and have been playing since 1974. Most of my original players have seen gone onto life in general and don't play anymore and for the last 25 years, I have been playing with groups in which I was the oldest person.
Significantly older now. As in the players are young enough to be my kids. The age gap between myself and the youngest person is 37 years.
Age isn't a issue at all and never came up in any of the my groups (and I been in many over the years); except for a parent of one of the players in one group who had concerns about her son playing a game with a guy three times his age. Once she met me, things were fine.
Odd that age is an issue with you. Having played with hundreds of people, never once has any player mentioned anything about age or how old I was.
I suspect it is because of love for the game. Age doesn't matter nor should it, IMO.
About the weirdo thing - define weirdo. Because I'm a weirdo in my town because I am the old grumpy guy that plays games (not just D&D) with kids and that makes me a weirdo amoung my peer group. I'm the weirdo because I jump out of perfectly good aircraft as a hobby. I am seen by the neighours practicing swordmanship (fencing / kendo) in my backyard on pleasant summer days. I make swords and knives (took a course in blade making which I still occasionally do when I can access the equipment).
By the standards and activities of my age group in my community, that makes me a weirdo with a capital W and in spades.