+ Log in or register to post
Results 1 to 6 of 6
Sunday, 3rd July, 2011, 07:17 PM #1
4 hours w/ RSD: Get Some Feedback
Get Some Feedback!
The bulk of this month’s column is going to talk about development choices and marketing for tabletop RPGs. There is an important part that relates to our objective of getting more fun out of the 4 hour game session, so I’ll lead with that and those of you uninterested in the business analysis won’t have to wade through it to get to the good stuff.
Before anything else happens at your table, a critical decision is made: What game to play. There are so many choices across dozens of genres and many versions and revisions. My advice is to start with a sample of options and see what your group really wants to play rather than making a choice for them and then trying to build consensus.
Send an email to your group with 3-5 options of games you’d like to run and ask if there’s any significant preferential choice. Remember to also ask if there’s a game people really don’t want to play – knowing about the “don’t wannas!” is as important as the “hell yeses!” If there doesn’t seem to be a consensus, try a different selection of games and repeat until you find something that gets your group really enthused.
If you’re playing an “open table” (so much more about this is coming in future columns but I’ll just put this comment in this column today for future reference), have a very clear description of your game available on an internet resource so that prospective players can understand what you’re offering before they arrive to play.
The overall suggestion here is to let the group lead (or self-select) to a game that they’re going to enjoy playing. It’s a simple thing, but surprisingly powerful in terms of generating more fun and more long-term interest – both of which tend to keep groups cohesive in the face of the forces that tend to break them up.
Diversity & Choice in the TRPG Market
The idea for this column started with a post on RPG Pundit’s blog (“Why do Commercial RPGs Fail”). Pundit listed 5 examples of common failures in design & marketing, but I felt that he was missing the most crucial factor: The market just doesn’t want what the publisher is selling.
A bit later, Greg Christopher responded on his blog (“What the Consumer Wants”) taking the position that designers need to ignore “market research” because using it to make design decisions produces crappy results.
I responded on his blog that he was confusing two different kinds of research – the kind where you try to divine preferences from consumer surveys and the kind where you try to determine if consumers would buy the product you are offering to sell.
The distinction is important, and fills the bulk of this column. The former (as I noted in my blog comment) does often create crappy results. The latter, on the other hand, should be a critical tool in your toolbox if you want to create and sell products to the hobby gaming market.
A caveat: If you are interested in creating art – that is, a work you create from your own passion to fulfill your own need to create, without a requirement that the work have commercial value, most of the following does not apply to you. Artists pursuing art for art’s sake are free from the constraints of having to worry if they can monetize that work. The success or failure of their labors is simply internal to themselves and requires no validation in the form of sales (or even other people using or appreciating the work). On the other hand, if you think “someone will buy and play this thing I’m making”, you’ve defined your work as something that does therefore fall into the realm of commercial endeavor and thus is subject to the judgment of the marketplace.
Why Asking People What They Want Doesn’t Work
The first kind of market research tends to fail miserably. If you conduct research that asks people a bunch of questions about their preferences for something like a game design and a genre, and then try to make a product that maps to those answers regardless of your opinion of them, you’re almost certainly doomed to fail.
The reason is that there’s a known bug in human psychology that limits most people’s ability to self-analyze their own preferences. Malcolm Gladwell wrote a book about this (which is fantastic), called [ame="http://www.amazon.com/Blink-Power-Thinking-Without/dp/0316010669"]Blink[/ame]. In short, when you ask someone “what do you like about ‘X’”, most people give you an answer that isn’t correct. When their conscious mind attempts to make sense of their unconscious desire, there’s some kind of disconnect which produces garbage output.
It is possible to get useful data out of such studies but they have to be crafted differently than you might expect. Instead of asking straightforward questions, you have to build a study that gets at people’s underlying desires with questions that cross-reference each other, and questions that help disqualify responses that are red herrings. And you can’t take the results in an unprocessed form either. Instead you have to analyze the result looking for clusters and patterns, and then try to tease out places where the data suggests there’s a need for something that the respondents are seeking. This intersection is called a “need/gap”, and they’re very valuable especially if you can figure out how to deliver a commercial solution to that need/gap.
These kinds of studies are beyond the scope of this column. I’ll end this section by just saying that I agree with the surface conclusion that if you try to make something commercially successful based on simplistic market research, you’re more than likely to fail.
Succeeding Without A Map
Most of the successful tabletop RPGs weren’t built around any grand theory of need/gaps, or in fact any real research of any kind. They are all mostly the result of individual creators bringing their ideas to the market and finding success.
If you were to simply use that as a guideline, you’d conclude that the strategy for success in this industry is to self-publish your game getting as little input as possible from others and wait for a market to find you. And you’d be wrong.
The problem is that what you’re seeing is the output of a ruthless process of winnowing. You’re only seeing the games that “got lucky”, in the sense that the designers produced a work for which a need/gap actually did exist, at a time when the market was interested in that topic, with at least adequate design, and a good enough brand to grab enough interest among early adopters to create the seed of a social network around which a community of players could aggregate.
You don’t see the hundreds (thousands probably) of games and designers that followed the same path and ran into a brick wall.
It’s hard to visualize this for tabletop RPGs because most of them never see enough traction to get into the public record. But the collectible card game market had a remarkable opportunity to capture this phenomenon. In 2001 Krause Publications printed the Scrye Collectible Card Game Checklist and Price Guide book. This book documented 100 different CCG games that had been created from 1993 to 2000. By 2003, only 3 of those games were still being produced: Magic: The Gathering, Vampire: The Eternal Struggle, and Legend of the Five Rings. In 2011, only 2 of those games are still being produced – VTES is again in a production hiatus.
Were those other 97 games all crap? No, there were some very good games in that list. But a very large percentage were not good fits for the market. They were genres too small to attract a viable audience. They were variations on an over-done theme. They were mechanically poor. They were boring. They exploited a license but had little to offer beyond the licensed IP. The reasons those games did not become evergreen products are legion.
When you look at the tabletop RPG market and see successful games produced without the appearance of any backup data, you’re seeing the tiny handful that escaped all the traps that caught their peers. You’re not seeing a success strategy. You’re seeing lotto winners.
The Simplest Test
Even some of those lotto winners used a bit of research to help them succeed. This is the easiest thing you can do if you’re a creator and want to get a sense of the value the market might have for the game you want to create.
Pay an artist to do an illustration that depicts the unique selling proposition of your game. You’d be surprised at how reasonable these prices can be – there are many more quality artists than there are regular commissions working in the hobby game industry. The smart ones appreciate the opportunity to connect with a creator at the earliest stages of the work – they know that downstream there may be substantial benefits to being a part of that early circle.
Once you’ve got your illustration, take it to game stores and to gaming conventions, and ask people a simple question: “Would you be interested in playing this game?”. Take notes. Don’t rely on your memory (which will play tricks on you). You’re looking for a substantial supermajority of positive responses. If you’re getting that kind of overwhelmingly positive feedback, chances are you’ve got yourself a good idea worth the effort to develop to the next stage. If you’re getting mixed reactions, it’s likely you don’t. Remember that a number of people telling you they like the work are just being nice – every “yes” comes with hidden caveats.
Let’s play a little game and pretend we’re seeing a series of these presentations. Check your own responses to the following images. “Would you want to play this game?” Also try “Tell me what this game is about from this picture”.
Here’s the classic that launched the genre. Any question about what this game is about?
But what about this image? What’s this game about? Would you play it?
Alternity was created by some of the best designers in the industry and was published by the biggest company in tabletop RPGs, yet it failed to find an audience. I was a retailer (and a publisher) when Alternity was announced and I remember thinking that I didn't understand what the game was supposed to be, and what audience it was supposed to address. And I formed that opinion from little more than the artwork displayed above - a sentiment that proved nearly universal within the market when the game was printed.
Most people who are enthusiasts in the hobby have a well-trained critical eye for games and can quickly sort them into mental categories of likely to play, or not likely to play just from single images.
If you listen to what they tell you, you can save yourself a lot of heartache. Chances are you’re going to get a lot of negative feedback. Great ideas are a true rarity – great ideas that are commercially viable are an even more exotic species.
Spend a few hundred bucks on an illustration, and save yourself tens of thousands of dollars and hundreds of hours of design chasing a dead end. Do a little bit of testing to help guide your ideas towards something with a pre-existing customer interest and you’ll avoid a lot of heartache.
What Great Designers Do
In my experience, great designers always have a portfolio of ideas they’re working on. They’re constantly showing them to people, getting feedback, making changes, and evolving those ideas seeking that elusive “YES!!!!” moment when they make that critical connection.
Rather than taking just one design and obsessing over it in the face of unlimited negative feedback, great designers work in parallel with many different ideas. They know that the best way to improve their odds of hitting the lotto is to buy a lot of tickets, not to play the same numbers over and over.
Ideas are the cheapest resource of our industry. They are commodities with little intrinsic value. Until you can validate an idea as being resonant with a substantial audience, you haven’t begun to move forward towards something that can be successfully published.
Clever designers are the constrained resource in this industry. Clever designers spew ideas like geysers – being in their presence is to be deluged with ideas for games. These people don’t obsess about any one of their ideas – they’re able to separate their cleverness from the reaction of others to those ideas and keep producing new ones as they seek a good fit.
Once you’ve put yourself through this process for a couple of years you’ll start to feel the unseen rhythms of the customers and you’ll start to get an innate sense for the places that an interesting design may find fertile soil. You may get lucky on the very first day, but that would be an extremely exceptional outcome. It’s more likely that you’ll work for a long time trying and discarding many ideas before you start to get significant overwhelming positive reactions. If the world worked any other way we’d be drowning under an ocean of great and successful games. The lack of that ocean is proof positive that great ideas with commercial value are truly exceptional.
--RSD / Atlanta, June 2011
Monday, 4th July, 2011, 05:14 PM #2
Acolyte (Lvl 2)
Excellent article, Ryan.
As someone who is in the "art for art's sake" category, we are destined to disagree on a few things because we have different goals, but I like that you recognize the distinction there and realize the logic breaks for that person.
Monday, 4th July, 2011, 06:50 PM #3
Minor Trickster (Lvl 4)
- Join Date
- Feb 2005
- Champaign/Urbana, IL
- Read 0 Reviews
ø Block KnightErrantJR
At the very least I think it is useful to keep in mind that listening to what people don't like is useful. Listening to what you have done that people do like is useful. Trying to get general ideas from people about what they think they want to see in the future is tricky and doesn't always work out the best.
Monday, 4th July, 2011, 07:00 PM #4
Pit Fiend (Lvl 26)
One obvious example of that is asking customers how you should price your product. Obviously, the ideal for the customer is that the product be free; and customers will typically claim the most they'd pay for something is lower than the amount they actually - when it's put to the test in the market - are willing to pay.
What customers say they want don't necessarily form a baseline criteria for design. If the customers say they want an 800-page full-colour book, but the market shows they would actually happily buy a 600-page black-and-white book, you have to at the very least question the wisdom of spending the extra money on interior colour.
Problem this, a lot of this info is clearer in hindsight than it is in advance.
Wednesday, 6th July, 2011, 05:30 AM #5
Minor Trickster (Lvl 4)
- Join Date
- Sep 2002
- Houston, TX
- Read 0 Reviews
ø Block Terramotus
Interesting article. And while I don't find any fault with any of the analysis (the picture trick seems like a pretty good idea to gauge if there's a market in an established genre), I do find fault with what I saw to be the implication of the article - that it IS in fact possible to market research your way into a successful product.
In fact, given his examples of the CCG market, forget the idea of success without research being lotto winners - only 2 products surviving after a decade ARE lotto winners, regardless of whether they did their market research properly.
I think that, often, market research fills a role of "helpful, but not sufficient". I won't even call it "necessary". There are a few reasons for this.
First, much of the creative process is, in fact, a winnowing of the wheat from the chaff idea-wise. Do I need to do research to know that an RPG about dogs playing poker is probably not going to be a best-seller? Probably not.
Second, much of the market research falls into the role that, increasingly, criminal profiling is filling. That is to say that successes and failures fit the mold that the research would predict after the fact, but that its predictive power is far from certain. Of those CCG games previously mentioned, I'm sure that a thorough analysis could elaborate in great detail why the many failures failed, and why the successes suceeded, but if you looked at everything beforehand, and had no knowledge that Magic was already a successful product in 2001, would it really be possible to predict with any degree of certainty that Magic would be the one that came out on top through research? I suspect not.
Finally, and I think, most compellingly, I believe that many of the biggest successes in any industry are those that fill an "unrecognized need". They seize on the zeitgeist of the time and distill it in such a way that it's simultaneously exactly the way people were thinking and transformative at the time time. The success of D&D itself would not have been possible to predict. And while, in a mature market, where transformative products are unlikely to occur the accuracy of market research is greater, in the long term, it's those transformative products themselves that tend to stay evergreen, for a variety of reasons.
Ultimately, I would say that his article somewhat misses the point. Many of these transformative products were the product of a singular vision. In fact, I think that, in fact, you DO need to maintain that clarity of vision, which sometimes needs to be a situation where input from others is avoided. Design by committee can work, but only in a very small committee. More often, the dilution of the vision serves to ruin the product.
I understand the goal of market research, though. From a comapny's perspective, how do you make sure that these transformative products are produced by employees of your company and not by either competitors or upstart companies?
The short answer to that is that while you can do certain things make it possible, you really can't ensure it with any kind of accuracy. Do some shops catch lightning in a bottle more than once? Absolutely. Almost never do they realize what it was they were doing right, though, or are even able to put it into words. Even observing success stories you have too small a sample size for any kind of real conclusions, and are likely to suffer from confirmation bias and the placebo effect. If there were an actual solution to this problem, we'd have a very different landscape than the bajillions of contradictory management books on the market.
Last edited by Terramotus; Wednesday, 6th July, 2011 at 05:34 AM.
Monday, 11th July, 2011, 11:59 PM #6
Novice (Lvl 1)
This has generated a very interesting afternoon of reading for me. I want to say thank you to Mr. Dancey and the two bloggers mentioned as well as everyone who took the time to comment. I do have a couple of further questions, based partially on the articles and partially stemming from tangential thoughts.
What is the definition of a "commercially successful/viable" game or great idea? I would assume one of the basic underpinnings of the definition is the game must sell - but in what quantities, where and to whom?
In considering the development of a competing game in a given genre by segmenting the genre, what role do the actual mechanics of a given game play in segmenting a genre?
By RyanD in forum Roleplaying Games General DiscussionReplies: 51Last Post: Saturday, 19th February, 2011, 05:33 PM
By Michael Morris in forum Roleplaying Games General DiscussionReplies: 79Last Post: Thursday, 26th June, 2008, 04:59 AM
By jmucchiello in forum MetaReplies: 39Last Post: Tuesday, 1st February, 2005, 06:26 PM
By alsih2o in forum Archive-threadsReplies: 61Last Post: Monday, 17th May, 2004, 10:58 PM
By Shadowdancer in forum Miscellaneous Geek Talk & Media LoungeReplies: 8Last Post: Tuesday, 16th December, 2003, 10:22 AM