A couple of good points on pricing have come up.
If you charge a group rate, that implies a group is trying to book you. If there's a group in existance already, they would be HIGHLY motivated to just do it themselves, rather than pay you.
This means your better target audience is individuals who want to game, but have no group (don't know anybody). What you'd be selling is your ability to GM and put together a group of people.
Since size of group is your problem, not theirs, and the larger the group, the harder it is, stick to charging individuals. If I play in your game, charge me $X. It's not my business that there were 3 or 5 players at the table.
I'd also recommend developing a simple pricing scheme, say a flat rate of $X for a session, and then make sure your adventure was written to take 4 hours. That may mean speed GMing, it may mean giving them a bit of extra (you spend 5 hours with the group, but only charge them for 4 hours).
This would be kind of like how a band might charge $100 a man to play 3 sets in 3 hours, but they spend an hour setting up before the gig, a half-hour tearing down, and travel time. Basically, you deliver a product that an approximate amount of time, but you get paid a flat rate.
For the actual price you charge, you factor in expenses, time spent prepping and working, and compare the time spent versus how much you'd make at your other job (say minimum wage of $6 an hour).
Let's use these numbers:
5 hours adventure writing
5 hours materials making (props, NPCs, hand-outs, etc)
5 hours running a game (4 hour planned, 1 hour of stretch)
That's 15 hours, which working at minimum wage would be: $90
Ideally, you'd want to make $90 minimum to run this a session.
From there, you need to figure out what to charge, you could hit the whole group for 90, but frankly, I'd be mad if I had to pay more, just because YOU couldn't fill all the seats. This is why group pay sucks.
If you optimistically assume the D&D 4 PC party standard (basically, no game if you can't get 4 people), then you'd just divide the $90 by four, and charge each player $22.50. If less show up, you'll have to run some helper NPCs to fill the party and take the pay hit. If more show up, you will be rewarded for your work with a bonus.
I encourage you to take the math I used, and plug in your own values. How much time does it take to prep a game? How long will it take to run a session. Add them, and multiply by the Hourly pay you could reasonably have been making. Then divide by the number of players you realistically expect to have (low-ball it)
The result is the MINIMUM you should charge. You also need to be careful of charging too much, otherwise folks will simply decide its not worth it. Folks can be spendy or tight, it depends on how much they value your product.
Also be aware, that if the minimum $ you calculate sounds too high, that's a warning sign that this is not a viable product to make money on.
The above exercise gets you a minimum price, compared to alternative income sources (basically, a different job). It's the minimum price you're REQUIRED to charge per good economic sense, otherwise, you should be doing something else.
The other math to do is the optimistic math, of how much do I need to charge per your DESIRED, for your target income. After completing the first exercise, you now know how much time each session will take.
You'll want to know the maximum number of sessions you can run (per week), as well as practical levels. In my example, it takes 15 hours per session. Across a 40 hour work week, that's basically 2 sessions a week, maybe 3. Let's pretend 3, and multiple that by 50 weeks (2 weeks off for holidays, because while you may want to work your customers aren't around), I get 150 sessions per year.
Now let's figure out how much money I want to make from this venture. Federal poverty level is $12,000, let's pretend that's my goal.
That means, each session needs to bring in at least $80 (12000/150)
That's actually close to what the other method gave.
However, $12,000 for a full time job is pretty lousy. If we up that to what some starting teachers get (one of the lowest paid "educated" jobs), that's 30-40 thousand. Let's pretend $30,000 because we're not greedy.
You'd need to charge $200 a session. And that's assuming you are FULL time running sessions.
The fact of the matter is, anybody in a self-employed service business (say computer repair, plumbing, etc) is never working full time. There's dead time, driving time where there's no billable work time (which for you, just like a plumber, is time on site, doing the job). This is why they charge a lot of money. Because the customer is not only paying for the time onsite, but also the dry-spell between jobs. Otherwise, you'd go hungry.
This means, you've can't assume a full time load of 3 sessions per week. You've got to assume a slower stream of jobs, and that's just to get realistic estimates. Let's scale back to 1 session a week (because you've assembled a regular group who meets every saturday).
That's 50 sessions for $30,000, which in turn means charging $600 per session, or an assumption of $150 per player in a four player group.
Using my base math of 15 hours of work per session, to make $30,000 a year, assuming a decent level of business, you'd need to charge a minimum of $22.50 just to break even, and $150 per player just to ensure that you met a salary goal.
Just so you know, using the minimum wage comparison of $90 per session times 50 sessions per year is $4,500 which is NOT a lot of money when spread over a year.
Whereas $6 an hour times a 40 hour work week times 50 weeks = $12,000
Also, assume that taxes will be 40% of your income. It'll be less than that, but it's the safe way to make sure you don't get surprised by all the kinds of taxes you'll need to report (federal, medicare, SS, state, local, etc).
I would be curious for you to use my math, with your numbers, and post them here.
That'll tell us what your estimates are and show product viability