Max Roll­wage
February 2020

What do you work on?

I am working on the cog­ni­tive and neural mech­a­nisms that enable people to revise their beliefs/decisions when pre­sent­ed with new evi­dence. More specif­i­cal­ly, I am inter­est­ed in sit­u­a­tions where people seem unwill­ing to revise their beliefs, as exem­pli­fied by people with radical polit­i­cal or reli­gious beliefs.

“We hypoth­e­sized that radical beliefs might be asso­ci­at­ed with a domain-general deficit in rec­og­niz­ing and revis­ing our own mistakes.”

What did you do using Gorilla?

We inves­ti­gat­ed the cog­ni­tive under­pin­nings of radical (polit­i­cal) beliefs. Specif­i­cal­ly, we hypoth­e­sized that radical beliefs might be asso­ci­at­ed with a domain-general deficit in rec­og­niz­ing and revis­ing our own mis­takes. This would indi­cate a problem with metacog­ni­tion, rather than “first-order” task performance.

We used Gorilla to imple­ment this study, as our research ques­tion required large cohorts of par­tic­i­pants, coming from a diverse demo­graph­ic back­ground and having a wide range of polit­i­cal opin­ions. The Gorilla Code Editor was most helpful in allow­ing us to imple­ment and host bespoke behav­iour­al tasks online, without man­ag­ing our own server.

We used per­cep­tu­al tasks to measure the insight people have about the cor­rect­ness of their deci­sions. In our tasks par­tic­i­pants had to judge which of two patches con­tained more flick­er­ing dots, before rating their con­fi­dence in this deci­sion. Here, a strong cor­re­spon­dence between con­fi­dence ratings and actual accu­ra­cy (i.e. high con­fi­dence when correct and low con­fi­dence when incor­rect) indi­cates good metacog­ni­tive ability.

We com­bined these mea­sures of metacog­ni­tion with ques­tion­naire mea­sures about pol­i­tics and espe­cial­ly radical beliefs, e.g. the intol­er­ance people hold against oppos­ing polit­i­cal opinions.

What did you find?

As pre­dict­ed, we found that indi­vid­u­als holding radical beliefs dis­played a spe­cif­ic impair­ment in metacog­ni­tive abil­i­ties about low-level per­cep­tu­al dis­crim­i­na­tion judg­ments. Specif­i­cal­ly, more radical par­tic­i­pants dis­played less insight into the cor­rect­ness of their choices, and reduced updat­ing of their con­fi­dence when pre­sent­ed with addi­tion­al evi­dence (i.e. seeing the patches with dots again before rating their con­fi­dence). Our find­ings point to a generic resis­tance to rec­og­nize and revise incor­rect beliefs as a poten­tial driver of radicalization.

Did you include any special fea­tures in your study to ensure good quality data? If so, what did you do?

We include catch ques­tions in ques­tion­naires (e.g. “If you have read the ques­tion please answer with: Agree com­plete­ly”) to screen out people that answer ran­dom­ly. We also set a cri­te­ri­on for behav­iour­al task per­for­mance to make sure that people were paying atten­tion to the task.

More gen­er­al­ly, we incen­tivized people by paying them bonus money accord­ing to their task per­for­mance. I suspect that incen­tiviza­tion is the most crucial factor to ensure engagement.

What are the main ways people mis­un­der­stand your thesis?

In our recent study, we show that people with radical polit­i­cal beliefs show reduced insight into the cor­rect­ness of their deci­sions. This effect is present on both extremes of the polit­i­cal spec­trum. When I present those results, people often equate “radical” with “far right” (and ignore the far left) which is not what we find. There might be some selec­tive lis­ten­ing going on when people inter­pret our findings!

What real-world problem do you see that your research could impact?

In my eyes, under­stand­ing soci­etal polar­iza­tion is a crucial goal and there­fore I believe my research has direct rel­e­vance for an impor­tant issue of our time. Under­stand­ing the cog­ni­tive under­pin­nings of radical beliefs will hope­ful­ly help us to coun­ter­act the process of rad­i­cal­iza­tion in the future.

“Online research might be a game changer in solving the repli­ca­tion crisis.”

Has this study been published?

The study has been pub­lished in Current Biology here.

How do you think online research is going to change your field?

Online research will have a huge impact on the field of psy­chol­o­gy and cog­ni­tive neu­ro­science. In par­tic­u­lar, online research might be a game changer in solving the repli­ca­tion crisis. For instance, since it is rel­a­tive­ly easy, fast and inex­pen­sive to acquire online data, we have started to run inter­nal repli­ca­tion samples for all effects we find in our online studies. This helps to estab­lish the robust­ness of our effects. Such an approach could move the field towards more repro­ducible findings.

What is the biggest advan­tage of online research methods?

The pos­si­bil­i­ty to acquire large data within a short time. The samples also tend to be more rep­re­sen­ta­tive and diverse than the samples we have usually in the lab.

How did Gorilla make your life or research better, easier or faster?

For pilot­ing, it is great that the task builder pro­vides an inter­face with which it is pos­si­ble to create and test an experiment within a few hours/days. The ability to conduct complex behav­iour­al exper­i­ments online makes it pos­si­ble to acquire large data sets in very little time.

For you, what is the stand-out feature in Gorilla?

The task builder is very helpful for design­ing quick and easy behav­iour­al experiments.

What improve­ments would you like to see in Gorilla to make your research easier?

I think it would be great to have more inte­gra­tion between the task builder and the coding inter­face. I might not be up to date on this, but my expe­ri­ence is that using the task builder is fast and easy but under­stand­ably restrict­ed in scope, whereas coding a task from scratch takes a lot of time, but is very flex­i­ble. If it would be pos­si­ble to combine the best out of both worlds – for instance allow­ing edits to code created in the task builder – that would be extreme­ly helpful!

Response from Gorilla:

Hi Max,

We com­plete­ly agree! It’s already pos­si­ble to sup­ple­ment the task builder with script (see exam­ples here). But, we want to make this even better. We’re making some under­ly­ing archi­tec­ture changes to the task builder at the moment to make this possible.

Next year (2019), we expect there will be more options for users to add func­tion­al­i­ty to the task builder. In par­tic­u­lar, your two requests: (1) on the fly adap­tive trial gen­er­a­tion and (2) stimuli (visual and audi­to­ry) gen­er­at­ed in script.

When you’re not working, what do you enjoy doing?

I am a sports enthu­si­ast, espe­cial­ly enjoy­ing outdoor sports like climb­ing and hiking.

What science book have you read recent­ly that you’d rec­om­mend to others?

Surfing Uncer­tain­ty by Andy Clark

Max Roll­wage
Cognitive Neuroscience: Decision-making, metacognition, computational modelling
PhD student
Wellcome Centre for Human Neuroimaging
Max Rollwage

Ready to get started?

More Spotlight Interviews

Mircea Zloteanu

Mircea Zloteanu

[get-spotlight-info] "We showed that not all “posed” emotions are created equal, and that research needs to consider this for us to have an accurate picture of human emotion recognition." Continue Reading Mircea Zloteanu

Anqi Hu

Anqi Hu

[get-spotlight-info] “Online research has the great potential for us to reach out to a more representative sample and to create a more participant-friendly research experience. It is tremendously helpful for studies that investigate individual differences and learning in children.” Continue Reading Anqi Hu