Gam­i­fied Research Series: Under­stand­ing Value-Based Decision-Making

In this inter­view Josh explains how he used games to add a real­is­tic social dimen­sion to his exper­i­ments and to make them more engag­ing for his participants.

Josh has worked for over 15 years in both indus­try (Glax­o­SmithK­line, Nielsen) and acad­e­mia (Trin­i­ty Col­lege Dublin, ETH Zürich, Royal Hol­loway Uni­ver­si­ty of Lon­don) inves­ti­gat­ing a range of top­ics includ­ing: neu­roimag­ing meth­ods, neu­roe­co­nom­ics and social deci­sion mak­ing, rule-based learn­ing and automa­tion, early mark­ers of cog­ni­tive decline in age­ing, and reward and moti­va­tion in autism spec­trum con­di­tions. He cur­rent­ly has 47 peer-reviewed pub­li­ca­tions and has won over £1.5 mil­lion in grant fund­ing.  He holds a BSc in Psy­chol­o­gy and PhD in Cog­ni­tive Neu­ro­science from Royal Hol­loway Uni­ver­si­ty of London.

There is a grow­ing trend for researchers in the field of psy­chol­o­gy to use games in their exper­i­ments. This inter­view is part of a series inter­view­ing pio­neers the field of gam­i­fied research who have used Gorilla to devel­op their exper­i­men­tal designs.

What is the back­ground behind the research that you under­took using Gorilla?

My research is about social deci­sion-mak­ing, but I often wor­ried that terms in social psy­chol­o­gy were quite vague and ill-defined. In fact, I’ve writ­ten a blog about this in the past (Know­ing me, know­ing you? Do Psy­chol­o­gists under­stand each other?), and in my opin­ion it’s a big­ger prob­lem for social com­pared to cog­ni­tive psy­chol­o­gy. To get around this I’ve often taken tried-and-test­ed tasks from value-based deci­sion-mak­ing and changed the social ref­er­ence (i.e., answer the ques­tion for your­self or a friend or a stranger). There are lots of fun and real­is­tic gam­bling games which have a solid math­e­mat­i­cal frame­work under­pin­ning play­er behav­iour, mak­ing it very clear what you’re inves­ti­gat­ing. They’re also incred­i­bly mal­leable as you can turn them into social tasks through some­thing as sim­ple as get­ting par­tic­i­pants to observe other play­ers or pre­dict some­one else’s choices.

What was your moti­va­tion for turn­ing to online research?

One of the things we noticed when doing these projects was that under­grad­u­ate stu­dents show very lit­tle vari­abil­i­ty. I think it’s the whole WEIRD (White, Edu­cat­ed, Indus­tri­alised, Rich, Demo­c­ra­t­ic) pop­u­la­tion prob­lem. We had 100 most­ly white, most­ly female, psy­chol­o­gy stu­dents all between 18 and 30 years of age do the task in the lab and we just found so much homo­gene­ity in the response profiles.

What I was real­ly inter­est­ed in was not just social deci­sion-mak­ing but in the indi­vid­ual dif­fer­ences and try­ing to under­stand why some peo­ple are more influ­enced by oth­ers. So I want­ed to move online to get to a wider, more diverse and rep­re­sen­ta­tive group of peo­ple to do my experiments.

And why in par­tic­u­lar did you turn to online games?

Well the idea of using a game was a fol­low up to the need to go online. One of the things that we’ve noticed with online exper­i­ments is that if you’re pay­ing peo­ple or recruit­ing stu­dents, that’s fine, but it’s not easy to recruit if you don’t have money or cred­its to offer stu­dents. It’s even hard­er if you need you need a spe­cial­ist pop­u­la­tion that are already exhaust­ed from psy­chol­o­gy stud­ies. You need to engage them and moti­vate them to be part of your exper­i­ments. This was where I thought, there is such a clear dif­fer­ence between hav­ing a nice ani­mat­ed game that does what we were look­ing for, ver­sus what I could build, which was just bor­ing coloured boxes which wouldn’t have felt any­where near as real­is­tic or engaging.

Some­times you can per­form social deci­sion-mak­ing exper­i­ments with real peo­ple and mul­ti­play­er games but what I did this time was to trick peo­ple into think­ing they were inter­act­ing with real peo­ple. That’s why the real­ism and fram­ing was so impor­tant. When it looks like a slick prod­uct, peo­ple are more like­ly to believe that it’s a real per­son they’re play­ing with.

What exact­ly did your task involve?

The task itself is quite a fun one. You have two decks of cards, a red deck and a blue deck. One deck is stacked with more “win­ning” cards than the other and play­ers learn whether the red or blue deck pays out more often. How­ev­er, each card is also worth a set num­ber of points. This cre­ates four basic conditions:

  • Low Risk/Low Reward (like­ly to win but not worth many points),
  • Low Risk/High Reward (like­ly to win and worth lots of points),
  • High Risk/Low Reward (not like­ly to win and not worth many points),
  • High Risk/High Reward (not like­ly to win but worth lots more points).

This cre­ates more real­is­tic deci­sions that include risk and estab­lish how many more points a play­er needs in order to take a gam­ble. This varies a lot from per­son-to-per­son depend­ing on how risk-seek­ing or risk-aver­sive they are. The game ele­ments are a lot of fun here as you get stars and fire­works when you win vs a harsh buzzer and smoke when you don’t. This made the game much more engag­ing and enjoy­able than the orig­i­nal where you didn’t get any of this kind of stuff.

The social aspect of the game comes in with a lit­tle image of a finan­cial advi­sor and a speech bub­ble in the bot­tom right cor­ner that gives you advice about which deck to choose. We var­ied the qual­i­ty of the advice and also the risks and rewards asso­ci­at­ed with the decks, with­in-sub­jects. We want­ed to see how much empha­sis peo­ple were putting on the social infor­ma­tion they were get­ting and how much of their deci­sion-mak­ing was based on past expe­ri­ences, as well as how their deci­sion-mak­ing was affect­ed by the volatil­i­ty of the advice and decks of cards. We tend to ignore advice in a sta­ble world, but if the world is con­stant­ly chang­ing then we pay more atten­tion to what other peo­ple have to say.


What fea­tures of gam­i­fi­ca­tion did you find most use­ful and is there any­thing else you would like to have done but couldn’t at the time?

There are so many facets to think about. Most gam­i­fi­ca­tion works with badges, points, and leader­boards, but what’s real­ly cool are the research games. The one’s where you don’t know you’re doing a task. These tend to be more enjoy­able and peo­ple will come back more often and rec­om­mend it to other peo­ple. Although we didn’t have to do it with this exper­i­ment, if we want­ed to do repeat­ed mea­sures and we want­ed peo­ple to come back and play the game once a week to see if there’s an improve­ment, you need to make it enjoy­able and fun if you want peo­ple to give up their time. There’s also eco­log­i­cal valid­i­ty to think about. In our sit­u­a­tion, we have a nice deck of cards and it looks a lot like some­thing you’d have from the app store, some­thing that peo­ple are engag­ing with in their daily life. The hope is that per­for­mance would be clos­er to what you’d expect to see in a real-world gam­bling situation.

In terms of stuff that I want­ed that couldn’t be done, I guess the next stage would be par­tic­i­pants deal­ing with real peo­ple. Some­thing like The Hive where you actu­al­ly have real mul­ti­play­er, but there wasn’t any­thing in this exper­i­ment that I thought couldn’t be done.

Would you include gam­i­fied tasks in your future experiments?

It does depend a lit­tle. There are a few sit­u­a­tions where I would go back to basic exper­i­ments, such as when you need peo­ple to be very focused on a spe­cif­ic item and the gam­i­fied fea­tures might dis­tract them from the task, like if you’re doing eye track­ing. You would also need to think about whether gam­i­fi­ca­tion actu­al­ly con­t­a­m­i­nates your results by chang­ing base­line moti­va­tion, or if you are actu­al­ly chang­ing some­thing in your exper­i­men­tal design. With­in all of my exper­i­ments, I always try to have with­in-sub­ject con­trol tri­als so you’re always com­par­ing against a base­line that is inbuilt into your exper­i­ment. So I don’t think that’s such a big prob­lem. In most cases, I def­i­nite­ly would use gam­i­fied tasks in the future.

This inter­view is part of a wider series of inter­views look­ing at gam­i­fied research. Make sure to fol­low this link to have a read through them!

Sid Prab­hu-Naik Sid is a PhD stu­dent based in the Depart­ment of Exper­i­men­tal Psy­chol­o­gy at UCL. He is work­ing part time with Gorilla cre­at­ing a suite of fun games to col­lect research data to bet­ter under­stand some of the cog­ni­tive mech­a­nisms behind lan­guage devel­op­ment. He is also look­ing at how aspects of gam­i­fi­ca­tion itself can con­tribute to more moti­vat­ed, atten­tive, and ulti­mate­ly suc­cess­ful learn­ing strategies.