research-gamified-gorilla
Gam­i­fied Research Series: An Excit­ing Future

In this inter­view, Jo Ever­shed talks to us about some of the excit­ing research that has been made pos­si­ble using gam­i­fi­ca­tion in Gorilla and shares with us her vision for the future of the field.

There is a growing trend for researchers in the field of psy­chol­o­gy to use games in their exper­i­ments. This inter­view is part of a series inter­view­ing pio­neers in the field of gam­i­fied research who have used Gorilla to develop their exper­i­men­tal designs.

What First Got You Inter­est­ed in the Idea of Using Games for Research?

We play quite a lot of games at work. We’ve been playing Among Us recent­ly, and Among Us is basi­cal­ly a riff of Were­wolf, and Were­wolf is a riff of what used to be called Mafia, and Mafia was orig­i­nal­ly a psy­chol­o­gy task. There’s quite a lot of board games out there that actu­al­ly riff off the psy­cho­log­i­cal lit­er­a­ture. Really well-designed board games often take one narrow type of rea­son­ing and turn it into an inter­est­ing game that’s nice to exploit and opti­mize. And people pay good money to buy these games to play on their own and with friends. With this in mind, surely there’s space to gamify psy­chol­o­gy tasks and make them fun for par­tic­i­pants without com­pro­mis­ing data quality?

At the moment, researchers mostly pay par­tic­i­pants to take part in their exper­i­ments, but an alter­na­tive approach is to make the exper­i­ments fun, so that par­tic­i­pants take part for free.

As an example, using Sea Hero Quest, researchers have record­ed data from 4.3 million players, who have played for a total of over 117 years. Col­lect­ing 117 years’ worth of data via a recruit­ment service such as Pro­lif­ic (117 par­tic­i­pants, 525600 minutes each at £7.50 per hour) would cost you over £10.7 million. It’s highly unlike­ly that a grant would have funded that. By apply­ing for money to create a game instead of par­tic­i­pant fees, the researchers have col­lect­ed their data at a frac­tion of the cost.

The more par­tic­i­pant hours you need, the more ‘budget’ you have for game devel­op­ment. For some exper­i­ments, it’s cheaper to pay par­tic­i­pants for their time, but for others, it’s a better invest­ment to create a fun-to-play game.

Game devel­op­ers who also under­stand behav­iour­al sci­ences are a niche within a niche, and so hard and/or expen­sive to find. Con­se­quent­ly, cre­at­ing games has been suf­fi­cient­ly costly that it was rarely a good invest­ment of research funds unless an extreme­ly high number of par­tic­i­pant hours are needed. This could either be driven by needing a huge number of par­tic­i­pants, or a smaller number of par­tic­i­pants tested or trained at several time points.

But if we could reduce the cost of game devel­op­ment, by pro­vid­ing tools that enable researchers to create their own games, then gam­i­fied tasks could become main­stream for online psy­chol­o­gy exper­i­ments. This is what we’re aiming to achieve with the Gorilla Game Builder. Unsur­pris­ing­ly, researchers care about their par­tic­i­pants’ expe­ri­ence of taking part in research. We want them to have a good enough time so that they are happy to come back and take part in other research.

Where Can Gam­i­fi­ca­tion Have the Most Impact?

Games are already used in a few spe­cial­ist areas of research. One such sce­nario is for lon­gi­tu­di­nal research. If you need to test someone five times a week for eight weeks, it’s going to be hard if the task is boring.

There’s no amount of money you could pay me to do a Go/No-Go task every day for 30 days, but if you design a fun and inter­est­ing game then maybe I’d take part. My feeling isn’t unique, lots of par­tic­i­pants feel the same way! Con­se­quent­ly, train­ing studies create a research require­ment that has jus­ti­fied the budget for games. If you can get good enough data quality and reten­tion of par­tic­i­pants in your lon­gi­tu­di­nal study, then the value of that data is so extra­or­di­nar­i­ly high that it’s worth that addi­tion­al investment.

A second place where games have already been used is working with chil­dren. If you try and get a child to do a dull, repet­i­tive, ugly task it’s both hard to explain to them what to do and you’ll lose their atten­tion. Chil­dren won’t take part for the good of science. If they aren’t enjoy­ing them­selves, they’ll simply stop par­tic­i­pat­ing and you won’t get your data. But kids will enjoy and be moti­vat­ed by quite a basic level of gam­i­fi­ca­tion. It doesn’t need to be a sophis­ti­cat­ed game. It just needs to be visu­al­ly appeal­ing; pretty and sparkly and rewarding.

A third place where games come into their own is for devel­op­ing edu­ca­tion­al inter­ven­tions. Gam­i­fy­ing learn­ing moti­vates stu­dents to keep trying, where a stan­dard train­ing task might lose their atten­tion. Games are a great envi­ron­ment for giving stu­dents prac­tice at exactly the right level of dif­fi­cul­ty to promote learn­ing. Not too hard that they fail, but not too easy either. We’ve worked on a couple of games with researchers; one was a game to teach deaf chil­dren how to speech-read, another was teach­ing addi­tion and sub­trac­tion to primary-aged stu­dents. There is huge scope here for devel­op­ing inter­ven­tions that stick, keeping users engaged and learn­ing for longer, while simul­ta­ne­ous­ly col­lect­ing high-quality research data about cog­ni­tive development.

What Research Is Needed on Gamification?

An inter­est­ing basic study that should be done in games is to find out what impact dif­fer­ent fea­tures have on the psy­cho­log­i­cal expe­ri­ence, includ­ing per­for­mance, atten­tion, moti­va­tion, attri­tion, enjoy­ment. There are obvious things like graph­ics, but also within the game you have reward fea­tures like par­ti­cles and sound effects. How impor­tant are those?

Also you have the visual “skin” as we call it. With Call of Duty for example it’s all tough man, dark greens, mil­i­tary and grungy with slight­ly ominous music in the back­ground. You could do exactly the same game in bright neons with kids in T‑shirts and water bal­loons or water guns, with all the ben­e­fits of atten­tion­al control and mental rota­tion. We actu­al­ly have a design for a version of a first-person shooter where you can manip­u­late the music, artwork, nar­ra­tive, and other dimen­sions of game­play, to inves­ti­gate the impact of each dimen­sion separately.

Parents across the globe worry about the impact of games on their children’s devel­op­ing brains. It would be great if there was more evi­dence of the impact playing games has on people’s psyche, and even the option to cus­tomise the look and feel of a game for dif­fer­ent age groups based on evi­dence, not ideology!

One inter­est­ing dimen­sion is emo­tion­al arousal. Researchers have found that they see more trans­fer to other tasks with people who play first-person shooter games than from people who play games like Tetris, which don’t have as much emo­tion­al arousal. An arm­chair spec­u­la­tion — that I think could be inter­est­ing to test — is whether a sense of threat leads to greater task trans­fer. If you feel that your “life” depends on getting better, maybe your brain responds to learn­ing dif­fer­ent­ly than when the expe­ri­ence is more like chewing gum for the mind?

What Do You Think the Next Big Devel­op­ments Will Be in Gam­i­fied Research?

Tech­ni­cal com­plex­i­ty and cost are big bar­ri­ers to inno­va­tion in any indus­try. But, if we can reduce the cost and tech­ni­cal com­plex­i­ty of cre­at­ing research games, then — given the gains above — I believe more researchers will want to use them. This is our aim with the Game Builder. In a nut­shell, by making it cheaper and easier for researchers, we hope to open up gam­i­fi­ca­tion to many more researchers.

What happens after that will be fas­ci­nat­ing! In my expe­ri­ence, remov­ing tech­ni­cal bar­ri­ers and there­fore empow­er­ing and lib­er­at­ing researchers to follow their curios­i­ty and fulfil their research ambi­tions has a big impact. Five years ago, online behav­iour­al research was still pretty niche. But easy-to-use and pow­er­ful tools like Gorilla helped it become main­stream. Perhaps the same thing will happen with games?

As we’ve seen, games have the capac­i­ty to answer ques­tions that were pre­vi­ous­ly too dif­fi­cult to inves­ti­gate. In par­tic­u­lar, the ease with which train­ing and edu­ca­tion­al inter­ven­tions can be designed and tested is extra­or­di­nary. If games become a main­stream research tool used in edu­ca­tion­al research, then we’re going to dis­cov­er so much more about general and indi­vid­ual devel­op­ment of cog­ni­tive process­es. Fur­ther­more, the knowl­edge and games created will bring researchers much closer to cre­at­ing inter­ven­tions that can have a pos­i­tive impact on people’s lives at scale.

We might also see an impact on career pro­gres­sion. At the moment, Ph.D. stu­dents tend to con­sid­er aca­d­e­m­ic research or uni­ver­si­ty teach­ing as their main options. This is unsur­pris­ing, as Ph.D. stu­dents have expo­sure to these careers, but fewer are exposed to oppor­tu­ni­ties in industry.

However, roles for behav­iour­al sci­en­tists are pro­lif­er­at­ing in indus­try. Yet, many roles involve leaving behind your subject matter exper­tise and apply­ing exper­i­men­ta­tion and data analy­sis skills more broadly, which can feel dis­heart­en­ing. A career pro­gres­sion that makes use of their spe­cial­ist knowl­edge might provide a more ful­fill­ing alter­na­tive.  Expe­ri­ence with gam­i­fied research may create a natural pro­gres­sion for Ph.D. stu­dents at uni­ver­si­ties into EdTech, FinTech, HealthTech, ResTech, or in fact a wide range of SaaS (Soft­ware as a Service) start-ups.

We’d still have the teach­ing, research, and general indus­try tracks, but now there’s another option, which is to take the actual subject and topic of your study into indus­try and see whether you can turn that into a product that offers some value to the world.

Of course, that will all need to be handled care­ful­ly to make sure that people aren’t cre­at­ing snake oil. But I cer­tain­ly think there’s a better hope of having good, evi­dence-based prod­ucts if they’re being devel­oped by people who spent three years study­ing the core science and working with those ideas. We’ve seen evi­dence of these prod­ucts in Health Tech. There are now many habit-forming apps helping users stop drink­ing or smoking, exer­cise or med­i­tate more, and even lose or gain weight. I’d love to see the same amount of evi­dence-based inno­va­tion reach­ing EdTech.

So, while we’re ini­tial­ly focused on pro­vid­ing the tools to set people free to do research, our hope is to enable sci­en­tists to create the evi­dence-based prod­ucts and ser­vices of the future that improve people’s lives.

Jo Ever­shed

Jo is the CEO and co-founder of Caul­dron and Gorilla. Her mission is to provide behav­iour­al sci­en­tists with the tools needed to improve the scale and impact of the evi­dence-based inter­ven­tions that benefit society. 

Sid Prabhu-Naik

Sid is a PhD student based in the Depart­ment of Exper­i­men­tal Psy­chol­o­gy at UCL. He is working part time with Gorilla cre­at­ing a suite of fun games to collect research data to better under­stand some of the cog­ni­tive mech­a­nisms behind lan­guage devel­op­ment. He is also looking at how aspects of gam­i­fi­ca­tion itself can con­tribute to more moti­vat­ed, atten­tive, and ulti­mate­ly suc­cess­ful learn­ing strategies.