Interesting Article on Beta Tester Motivation

Bug reporting and feedback only please.
Posts: 933
Joined: Fri Nov 01, 2013 8:43 am

Interesting Article on Beta Tester Motivation

Postby Hyperion703 » Tue May 02, 2017 5:56 am

I've done a handful of beta tests for similar products. Most have had a bit more tester feedback than we're getting so far for ELWG. I pondered this and did some research. I found a great article on how project managers can motivate the testers in their beta programs:

I could have emailed this to you, Paul, but I think it's aimed as much at the testers themselves as the lead manager, maybe more so. Posting it here seemed appropriate. Here's something of relevance:

"The beta starts and feedback rolls in. Enthusiastic, detailed feedback. Then, less. You coddle. You remind. You badger…

In general, you should expect that only a fifth of the beta testers will follow-through.

'Most beta testers will try out the program when they first get it, and then lose interest,' says Joel Spolsky. '[O]nly one out of five testers is really going to try the product and send you feedback.' He goes on to say that as a rule of thumb you should have 100 active beta testers per employee, and if the 1 in 5 ratio is to be believed, you should plan on recruiting upwards of 500 people in order to ensure 100 solid testers. 'So, for example, if you have a QA department with 3 testers, you should approve 1500 beta applications to get 300 serious testers. Fewer than this and you won’t hear everything. More than this and you’ll be deluged with repeated feedback.' The takeaway: you’re going to need to recruit a lot of beta testers."

FWIW, I think you're doing a hell of a job according to their five suggestions (good work!). I know you recruited a ton of testers (point number one). For the second motivator, I don't know how much control you have over it. Unfortunately, I think the mobile gaming community is a bit niche; some join betas in order to improve products or because they love your brand. But, more than most other types of products, gamers tend to join betas more for #3 of the second list, "Early access to your products". Admittedly this is one motivator for myself (though not primary), and not altruistic in the least. Rubicon has little control of the whims and desires of the gaming community at large. You proved you are dedicated to number three on the list, and are certainly a 'gracious beta testing host' by the way you dealt with Google after they tried to put the $10 hurt locker on us. I think number four, "acting on tester feedback" is probably the best I've ever seen yet; I'm seriously impressed by your flexibility and willingness to change aspects of the game based on tester feedback. And I think you do a good job (as well as an indie company can do) on point number five. If you did like it said and "In general, if you choose to compensate beta testers, keep the overall value of the incentive in line with the type of feedback they are providing," you might want to consider buying Abacist a car or something. :)

So, great job. You nailed pretty much everything on their list. Which is why I'm still confounded at the relative lack of tester feedback thus far. There's, like, four or five testers that are reporting regularly and I just don't understand why. At this rate, we might be able to hammer out a really well-tested product in about three months. I'm hoping, then, that it picks up...

Stay up,
Think Style. Think Skill. ThinkTank.
Dusty Nuggets Battalion: Named for the layer of dust accumulating on my HQ

User avatarRubicon
Site Admin
Posts: 4814
Joined: Tue Jan 03, 2012 11:13 pm

Re: Interesting Article on Beta Tester Motivation

Postby Rubicon » Tue May 02, 2017 8:06 am

Thanks. I think his percentages are off but then he's not talking about mobile. :)
Regards, Paul Johnson.

Posts: 153
Joined: Tue Mar 28, 2017 3:24 pm

Re: Interesting Article on Beta Tester Motivation

Postby Scotch21 » Tue May 02, 2017 1:59 pm

I agree w hype, think you guys are doing an awesome job and would like to that you and your team for all the hours/months of hard work to get this game out to us. I'm no programmer but I give the feed back I can. I can defiantly admit that part of my reason for joint the beta is early access, That being said it's is not the only reason, I do also want to help you all in any way I can.

Keep up the great work.

- Rusty Entente Defence (RED)

Posts: 324
Joined: Tue Apr 18, 2017 7:01 pm

Re: Interesting Article on Beta Tester Motivation

Postby Abacist » Tue May 02, 2017 3:07 pm

Thanks for the shout out hype. The greatest reward to me is that Rubicon is creating a kick ass game, and it will be an honor to have Jim "Abacist" Denton listed in the Beta Testers list.

Return to Epic Little War Game - BETA

Who is online

Users browsing this forum: No registered users and 1 guest