I think the initial cost is part of what makes people think it's expensive to start.  Even if the actual cost is low, there's an up-front cost that can feel imposing.  In that sense, starting an RPG hobby is more like starting a video game hobby than a sport or board game hobby.
Consider:
If you want to play a friendly, introductory game of bowling with your friends on a weekend, everybody spends a small fee to rent a lane and shoes.  You play a round.  Anyone who gets serious eventually buys their own shoes, ball, etc.
If you want to play a friendly, introductory game of basketball/soccer/rugby on a weekend, one person buys a ball.  Everyone else shows up at a park.  You play a game.  Anyone who gets serious buys their own ball, shoes, gear, etc.
If you want to play a friendly, introductory game of D&D with your friends on a weekend, often at least one person buys ~$150 worth of books (PHB, DMG, MM).  If people don't want to have to share PHBs, multiple people buy copies.  A bunch of people buy dice.  The startup cost to try the game out can easily be over $200.