Not so long ago, it seemed like the OGL was a green, wide open pasture. After the initial glut of d20 and OGL products, it looked like a healthy crop of third party publishers was left standing, with the field still welcoming to a small publisher with an idea. To be sure, sales were leveling off, but there still existed a definable segment of the market to which good products could be marketed.
Then came Dungeons & Dragons 4e, and with it, the end of Wizards of the Coast's involvement in open gaming. After a certain amount of foot shuffling, they finally produced the GSL, the next best thing to offering to set your business on fire and write a postcard of apology should Wizards find you in breach of contract. While 4e produced something of a split between 3e players and 4e players, the reactions of publishers were decidedly more complex. Many wanted a piece of the pie, others were upset their 3e lines had been undercut, and just about everyone wanted to know what was coming next. Did it mean the end of Open Gaming as a mainstream pursuit, or the true beginning?
So what can you do with the OGL now that 3e is yesterday's breakfast?
Third Edition, the Way It Was Always Meant To Be
Several publishers have had the notion that the open gaming community has effectively inherited support for 3e, and as a result, there has been a scramble to support that edition. So far, no one has pledged to simply act as if 4e never happened. Paizo has decided to turn their Pathfinder trademark into a new version of 3.5, using basically compatible stats while incorporating a lot of 4e-inspired concepts and a number of innovations suggested late in the game by Wizards products and other publishers, such as Mongoose. More powers, more options, more juice, but maybe a little refined and simplified. If you thought your game plan was to produce a 3.5 based game, but with at-will powers and other 4e notions, that ship has sailed. There is probably not a good way to compete with Paizo for that segment of the market. They were first, they are bigger than a basement publisher, so they win.
So, then, the option remains to make a 3.5 game that does the opposite. There is still plenty of room for a 3.5 variant that is basically recognizable, but trims out some 3e concepts that were perhaps less popular with AD&D players or players familiar with non-D&D games. You could make something fairly AD&D retro in style, or even look back to the Rules Encyclopedia for flavor and philosophical inspiration. Another approach might be to use variant material; leave 3.5 largely intact, but throw in options from Unearthed Arcana or any OGL book you fancy.
Finally, there is perhaps a market for a repackaged SRD with some minor errata and fixes, with a very low price point. In theory, there might be a market for a $1 PDF that amounts to D&D 3.51. I would at least be curious.
Embrace a Variant
Castles & Crusades, True20, and the Conan game all have their adherents. Runequest is a very different, non-d20 OGL option. But tapping into an existing customer base has its advantages. You wouldn't have to do all the heavy lifting yourself. In many cases, publishers would be extremely eager to have anyone else continuing to push their brand. Make sure your chosen publisher isn't prepared to drink the purple Look-Aid and that you will continue to have access to important Open Gaming Content should someone fold or sign a new contract with Wizards.
The disadvantage of this approach is that these games are losing a lot of their base over time as people convert to newer games. It would fall on you to a great extent to keep things fresh.
Get Creative
A final option is to simply create a new game, using whatever OGL you desire and making it work the way you think it should. You have the advantage that someone has already written a lot of the OGC for you to use, and a lot of the terminology will be familiar to new players, but let's be honest. You are essentially designing a game with its own fan base, which you have yet to create. It is, in the end, not much different than doing your own homebrew. The only reason you would do something like this would be if you have a specific design in mind that happens to use the OGL that you think will work. The OGL is a resource, not a reason for being, for a game that is not clearly compatible with other games. Mutants & Masterminds created its own niche, as did Castles & Crusades, but there is no guarantee you will do the same. And the market today is very different. Your audience will be sophisticated hobbyists who have probably purchased PDFs before. Most likely, you will publish electronically, with low production costs, and yet to be successful, you will have to demonstrate your commitment to high production values.
Be a Fan
The OGL leaves behind an unprecedented opportunity for fans of D&D 3e to continue producing new material in a "safe harbor." Many older games, including versions of D&D, still have a thriving fan community. With the OGL, 3e has a chance at more robust support, more publisher interest, and less legal troubles than other out of print games. Just because it does not provide a supplemental paycheck does not mean it is not worth doing.
There is nothing "unprofessional" about producing fan work or free work or work for insignificant pay. Sometimes, it is necessary to separate what we do because we can versus what we do because it pays. Before publishing a line of OGL products hoping to catch the remaining market, consider whether it might not be cheaper in the long run to release much of the material at very low cost and pinning your intended profits on something that makes sense now. Loss leaders can make a lot of sense in an industry where people often buy because they see a name they like. Further, there is something to be said for "business losses" incurred in producing real art, provided those losses are reasonable. I daresay that throughout this whole interesting adventure, more money has been lost trying to make money than was lost simply trying to write a damned good book.
In Summary
The timing of the 4e and the GSL was a demoralizing blow to people excited by 3e and the Open Gaming concept, but it is by no means the end of the road. Ultimately, it is about your games, your campaigns, and your chance to exercise your creativity. If you want to write or publish, your success depends on what you produce, not some nebulous, hype-driven industry you have no control over. If what you can produce now would have sold in the heyday of the d20 explosion, but you don't think there would be a market for it now, was it really that valuable a book? The key is to produce really good work, think about the audience you want to reach and what they value, and, for commercial endeavors, setting the price low enough for people to purchase a book on impulse or out of curiosity. Before publishing in paper, realize that the market has changed and the market is weak. Something new will have to truly stand on its own. There are already shelves and shelves of worthy OGL material still out there, unread and unpurchased.
Keep the eye on the bottom line. Your bottom line. What makes it worth it to
you?