Lies, Damn Lies, and Statistics


Hey! Long time no see! I've had this problem where I've been busy working on my business and haven't had a ton of time to talk about the industry as a whole. I hope to rectify that over the next month or so and dig into some topics that I've wanted to discuss, but my planned thought process has been interrupted by something I read recently.


Joe and his team called every US board game retailer they could find on Google, resulting in 1042 stores that sell games like Catan and Scythe. Out of those stores, 223 indicated that they support tabletop Kickstarter projects (21%). Of the remaining 79%, the reasons they weren’t interested in backing KS campaigns included:

  • Most said they would rather just wait until the game entered distribution.

  • They believed that if a person wanted a game, that person would just buy it on Kickstarter, not through the store.

  • Some had backed KS projects in the past and the games just sat on the shelf.

  • A few said that cash flow and cost were a concern.

 

This appeared in a recent blog post from Jamey Stegmaier, and it kind of got me a little hot under the collar. Polling is a science, a science which people study the best methods of, and then practice those best methods, in order to insure that their polls are meeting their needs. Polls can have various needs, whether we choose to acknowledge this or not. I can devise a poll whose only purpose is to provide confirmation bias for my own beliefs, and I can devise a poll whose purpose is the discovery of as much truth as I can get my survey respondents to share with me. We're smack dab in the early part of primary season, and there are numerous polls you can now find to see who is leading the Democratic Primary polls in various early-polling states. Because those polls are from places like Pew, CNN, and the Economist, they have lots of information about methodology and the questions asked right there in the polling results. From 14 JUL to 16 JUL, YouGov polled 1,500 members of an opt-in internet panel in a random sampling of voting age members; of which 1,149 were registered voters. The survey tells you methodology of selection, weighting of the voter groups based on age, race, gender, etc... Oh, just go read the darn thing, it's a mere 315 pages and a lot of fun to read... If you didn't completely understand all that talk about weighting the poll, there's an old Harvard Business Review post that explains the problems with modern political polling and bias pretty well. This industry suffers from a near complete lack of real information, factual and statistical. I desperately wish that we could fix this, but simple preferential polls "Who are you voting for in the upcoming Democratic Presidential primary?" cost thousands of dollars and take tens of thousands of phone calls to reach a statistically useful number. What's the point of all this?

I will play Viticulture anytime, anywhere.

It's that I like Jamey, and I love his games, but citing this "poll" infuriates me, and will until I see a list of everyone polled. Why do I want a list? Well, I talk to lots of retailers, and none of them took part in this poll to the best of their knowledge. Some of them think they got a phone call where they were asked for Catan, so it was probably a weekday in a game store, because we get that question every day. Do you want real information on Kickstarter? Do you want to know how many retailers are actively engaging in the platform, and truly discuss why some of them are not? Here's a script. "Hey, I'm (you should use your real name here) from (the name of a company you work for), and I was hoping I could talk to (primary decision maker for stocking purposes) about Kickstarter and see if they could help me gather some information." Now, to get a set of valid statistical data, we have to qualify the person who answered the phone. On a political polling call we ask a person their age, race, gender, registered political party, and the person they voted for in the last election, so we have statistical parity, or understand how to weigh the results if we can not reach statistical parity. Did any of that make sense, the HBR article above explains weighing results, go read it. How do we do this in the game industry? We begin with the simple truth that not all game stores are the same. There are many game stores who only sell Magic and other trading card games. They are no less than other game stores, but they are also not the market for your product, or most Kickstarter products. Some just do Magic and Warhammer, and they also may not be the best market for your product, or most KS products. So how do we find your market? "Can I ask about your product mix and your approximate turn rates on those products before we continue?" "Do you carry the Asmodee Top 40?" If yes, you're talking to someone who is at least making an effort in board games. If no, you might be on the wrong call, but we can continue to qualify them regardless. You just have to use the correct followup here. IF NO "What would you say the approximate wholesale value of your board game inventory is today, and your approximate turns of that inventory last year?" You have to decide what your threshold here is, to make the information useful to you. I have friends I respect who run 600 square feet of retail space with $13,000 in wholesale board game inventory, and friends who run 10,000 square feet that are so diverse their board inventory is about $20,000 wholesale. Both of those gentleman backed a Kickstarter in the last year, so you can't get too caught up in the exact number until you decide what's right for you. Of course, both of these gentlemen participate in the Asmodee Top 40 and wouldn't have been asked this followup, they'd have gotten the next one. IF YES "Did those Top 40 products combine for 120 turns in your store last year?" If yes, they turned each of those at least three times. It's a number that would depress me, but respectable enough to continue. If no, you might want to return to the followup question we used earlier, to gauge the width, depth, and engagement they have with board gaming. "Have you ever backed a board game on Kickstarter?" If yes, we get lots more questions. How many? How frequently? What drew you to those campaigns? If you want to make the questions open-ended and compile data, you'll get better data, if you want to provide pre-segmented answers you'll get good data, but not better data. You can ask "Did you back 1-5 KS campaigns in the last year? 6-10? 11 or more?" You can also just let people tell you why they backed something or you can offer them selections. You can ask people if price, deposit amount, payment terms, etc. etc. are why they chose to back something. Then ask those SAME people, that you know are backing KS campaigns, why they passed on some games they liked. For the people who don't back KS campaigns, ask them what the hurdles are to backing them. Then compile the data, showing you qualified each person you spoke to. We spoke to 1,680 stores, 1,042 that fit the sampling data we were looking for, and asked the following questions. Provide charts, share methodology. Anything less than full transparency is a push poll used to surround yourself with confirmation bias.

Our Recent Posts

Archive