Boy, do I sympathize. I'm in my forties, I teach full-time, I have a spirited little toddler, but our weekly gaming night with my wife and two of our buddies still happens maybe three out of four weeks. We get some takeout food, put the toddler to bed, and game just a couple of hours, enough for one combat and a little story advancement. Some nights, when we're dead beat, we just sit around and talk, but most of the time I just force myself to start DMing, my fatigue falls away, and I get swept up in the game. If one person out of the four of us can't make it, we switch the game to another night, or we simply don't play that week. We never play with just three.
Over the summer, we all took a vacation day on the same Friday, left our kid in daycare (she loves it and would have been going anyway if we'd been at work), and had an all-day game session. We might do it again around the holidays.
My suggestions for balancing gaming and middle age are:
--Relax and aim low in terms of how much you're going to accomplish
--Be flexible, schedule-wise
--Game with friends who would still enjoy hanging out with you even if you're too tired to game
--Use published modules or use your improv skills to cut down on prep time