Gam­i­fied Research Series: Under­stand­ing Value-Based Decision-Making

In this inter­view Josh explains how he used games to add a real­is­tic social dimen­sion to his exper­i­ments and to make them more engag­ing for his participants.

Josh has worked for over 15 years in both indus­try (Glax­o­SmithK­line, Nielsen) and acad­e­mia (Trinity College Dublin, ETH Zürich, Royal Hol­loway Uni­ver­si­ty of London) inves­ti­gat­ing a range of topics includ­ing: neu­roimag­ing methods, neu­roe­co­nom­ics and social deci­sion making, rule-based learn­ing and automa­tion, early markers of cog­ni­tive decline in ageing, and reward and moti­va­tion in autism spec­trum con­di­tions. He cur­rent­ly has 47 peer-reviewed pub­li­ca­tions and has won over £1.5 million in grant funding.  He holds a BSc in Psy­chol­o­gy and PhD in Cog­ni­tive Neu­ro­science from Royal Hol­loway Uni­ver­si­ty of London.

There is a growing trend for researchers in the field of psy­chol­o­gy to use games in their exper­i­ments. This inter­view is part of a series inter­view­ing pio­neers the field of gam­i­fied research who have used Gorilla to develop their exper­i­men­tal designs.

What is the back­ground behind the research that you under­took using Gorilla?

My research is about social deci­sion-making, but I often worried that terms in social psy­chol­o­gy were quite vague and ill-defined. In fact, I’ve written a blog about this in the past (Knowing me, knowing you? Do Psy­chol­o­gists under­stand each other?), and in my opinion it’s a bigger problem for social com­pared to cog­ni­tive psy­chol­o­gy. To get around this I’ve often taken tried-and-tested tasks from value-based deci­sion-making and changed the social ref­er­ence (i.e., answer the ques­tion for your­self or a friend or a stranger). There are lots of fun and real­is­tic gam­bling games which have a solid math­e­mat­i­cal frame­work under­pin­ning player behav­iour, making it very clear what you’re inves­ti­gat­ing. They’re also incred­i­bly mal­leable as you can turn them into social tasks through some­thing as simple as getting par­tic­i­pants to observe other players or predict someone else’s choices.

What was your moti­va­tion for turning to online research?

One of the things we noticed when doing these projects was that under­grad­u­ate stu­dents show very little vari­abil­i­ty. I think it’s the whole WEIRD (White, Edu­cat­ed, Indus­tri­alised, Rich, Demo­c­ra­t­ic) pop­u­la­tion problem. We had 100 mostly white, mostly female, psy­chol­o­gy stu­dents all between 18 and 30 years of age do the task in the lab and we just found so much homo­gene­ity in the response profiles.

What I was really inter­est­ed in was not just social deci­sion-making but in the indi­vid­ual dif­fer­ences and trying to under­stand why some people are more influ­enced by others. So I wanted to move online to get to a wider, more diverse and rep­re­sen­ta­tive group of people to do my experiments.

And why in par­tic­u­lar did you turn to online games?

Well the idea of using a game was a follow up to the need to go online. One of the things that we’ve noticed with online exper­i­ments is that if you’re paying people or recruit­ing stu­dents, that’s fine, but it’s not easy to recruit if you don’t have money or credits to offer stu­dents. It’s even harder if you need you need a spe­cial­ist pop­u­la­tion that are already exhaust­ed from psy­chol­o­gy studies. You need to engage them and moti­vate them to be part of your exper­i­ments. This was where I thought, there is such a clear dif­fer­ence between having a nice ani­mat­ed game that does what we were looking for, versus what I could build, which was just boring coloured boxes which wouldn’t have felt any­where near as real­is­tic or engaging.

Some­times you can perform social deci­sion-making exper­i­ments with real people and mul­ti­play­er games but what I did this time was to trick people into think­ing they were inter­act­ing with real people. That’s why the realism and framing was so impor­tant. When it looks like a slick product, people are more likely to believe that it’s a real person they’re playing with.

What exactly did your task involve?

The task itself is quite a fun one. You have two decks of cards, a red deck and a blue deck. One deck is stacked with more “winning” cards than the other and players learn whether the red or blue deck pays out more often. However, each card is also worth a set number of points. This creates four basic conditions:

  • Low Risk/Low Reward (likely to win but not worth many points),
  • Low Risk/High Reward (likely to win and worth lots of points),
  • High Risk/Low Reward (not likely to win and not worth many points),
  • High Risk/High Reward (not likely to win but worth lots more points).

This creates more real­is­tic deci­sions that include risk and estab­lish how many more points a player needs in order to take a gamble. This varies a lot from person-to-person depend­ing on how risk-seeking or risk-aver­sive they are. The game ele­ments are a lot of fun here as you get stars and fire­works when you win vs a harsh buzzer and smoke when you don’t. This made the game much more engag­ing and enjoy­able than the orig­i­nal where you didn’t get any of this kind of stuff.

The social aspect of the game comes in with a little image of a finan­cial advisor and a speech bubble in the bottom right corner that gives you advice about which deck to choose. We varied the quality of the advice and also the risks and rewards asso­ci­at­ed with the decks, within-sub­jects. We wanted to see how much empha­sis people were putting on the social infor­ma­tion they were getting and how much of their deci­sion-making was based on past expe­ri­ences, as well as how their deci­sion-making was affect­ed by the volatil­i­ty of the advice and decks of cards. We tend to ignore advice in a stable world, but if the world is con­stant­ly chang­ing then we pay more atten­tion to what other people have to say.

 

What fea­tures of gam­i­fi­ca­tion did you find most useful and is there any­thing else you would like to have done but couldn’t at the time?

There are so many facets to think about. Most gam­i­fi­ca­tion works with badges, points, and leader­boards, but what’s really cool are the research games. The one’s where you don’t know you’re doing a task. These tend to be more enjoy­able and people will come back more often and rec­om­mend it to other people. Although we didn’t have to do it with this experiment, if we wanted to do repeat­ed mea­sures and we wanted people to come back and play the game once a week to see if there’s an improve­ment, you need to make it enjoy­able and fun if you want people to give up their time. There’s also eco­log­i­cal valid­i­ty to think about. In our sit­u­a­tion, we have a nice deck of cards and it looks a lot like some­thing you’d have from the app store, some­thing that people are engag­ing with in their daily life. The hope is that per­for­mance would be closer to what you’d expect to see in a real-world gam­bling situation.

In terms of stuff that I wanted that couldn’t be done, I guess the next stage would be par­tic­i­pants dealing with real people. Some­thing like The Hive where you actu­al­ly have real mul­ti­play­er, but there wasn’t any­thing in this experiment that I thought couldn’t be done.

Would you include gam­i­fied tasks in your future experiments?

It does depend a little. There are a few sit­u­a­tions where I would go back to basic exper­i­ments, such as when you need people to be very focused on a spe­cif­ic item and the gam­i­fied fea­tures might dis­tract them from the task, like if you’re doing eye track­ing. You would also need to think about whether gam­i­fi­ca­tion actu­al­ly con­t­a­m­i­nates your results by chang­ing base­line moti­va­tion, or if you are actu­al­ly chang­ing some­thing in your exper­i­men­tal design. Within all of my exper­i­ments, I always try to have within-subject control trials so you’re always com­par­ing against a base­line that is inbuilt into your experiment. So I don’t think that’s such a big problem. In most cases, I def­i­nite­ly would use gam­i­fied tasks in the future.

This inter­view is part of a wider series of inter­views looking at gam­i­fied research. Make sure to follow this link to have a read through them!

 

 

Sid Prabhu-Naik

Sid is a PhD student based in the Depart­ment of Exper­i­men­tal Psy­chol­o­gy at UCL. He is working part time with Gorilla cre­at­ing a suite of fun games to collect research data to better under­stand some of the cog­ni­tive mech­a­nisms behind lan­guage devel­op­ment. He is also looking at how aspects of gam­i­fi­ca­tion itself can con­tribute to more moti­vat­ed, atten­tive, and ulti­mate­ly suc­cess­ful learn­ing strategies.