Max Roll­wage
February 2020

What do you work on?

I am work­ing on the cog­ni­tive and neur­al mech­a­nisms that enable peo­ple to revise their beliefs/decisions when pre­sent­ed with new evi­dence. More specif­i­cal­ly, I am inter­est­ed in sit­u­a­tions where peo­ple seem unwill­ing to revise their beliefs, as exem­pli­fied by peo­ple with rad­i­cal polit­i­cal or reli­gious beliefs.

“We hypoth­e­sized that rad­i­cal beliefs might be asso­ci­at­ed with a domain-gen­er­al deficit in rec­og­niz­ing and revis­ing our own mistakes.”

What did you do using Gorilla?

We inves­ti­gat­ed the cog­ni­tive under­pin­nings of rad­i­cal (polit­i­cal) beliefs. Specif­i­cal­ly, we hypoth­e­sized that rad­i­cal beliefs might be asso­ci­at­ed with a domain-gen­er­al deficit in rec­og­niz­ing and revis­ing our own mis­takes. This would indi­cate a prob­lem with metacog­ni­tion, rather than “first-order” task performance.

We used Gorilla to imple­ment this study, as our research ques­tion required large cohorts of par­tic­i­pants, com­ing from a diverse demo­graph­ic back­ground and hav­ing a wide range of polit­i­cal opin­ions. The Gorilla Code Edi­tor was most help­ful in allow­ing us to imple­ment and host bespoke behav­iour­al tasks online, with­out man­ag­ing our own server.

We used per­cep­tu­al tasks to mea­sure the insight peo­ple have about the cor­rect­ness of their deci­sions. In our tasks par­tic­i­pants had to judge which of two patch­es con­tained more flick­er­ing dots, before rat­ing their con­fi­dence in this deci­sion. Here, a strong cor­re­spon­dence between con­fi­dence rat­ings and actu­al accu­ra­cy (i.e. high con­fi­dence when cor­rect and low con­fi­dence when incor­rect) indi­cates good metacog­ni­tive ability.

We com­bined these mea­sures of metacog­ni­tion with ques­tion­naire mea­sures about pol­i­tics and espe­cial­ly rad­i­cal beliefs, e.g. the intol­er­ance peo­ple hold against oppos­ing polit­i­cal opinions.

What did you find?

As pre­dict­ed, we found that indi­vid­u­als hold­ing rad­i­cal beliefs dis­played a spe­cif­ic impair­ment in metacog­ni­tive abil­i­ties about low-level per­cep­tu­al dis­crim­i­na­tion judg­ments. Specif­i­cal­ly, more rad­i­cal par­tic­i­pants dis­played less insight into the cor­rect­ness of their choic­es, and reduced updat­ing of their con­fi­dence when pre­sent­ed with addi­tion­al evi­dence (i.e. see­ing the patch­es with dots again before rat­ing their con­fi­dence). Our find­ings point to a gener­ic resis­tance to rec­og­nize and revise incor­rect beliefs as a poten­tial dri­ver of radicalization.

Did you include any spe­cial fea­tures in your study to ensure good qual­i­ty data? If so, what did you do?

We include catch ques­tions in ques­tion­naires (e.g. “If you have read the ques­tion please answer with: Agree com­plete­ly”) to screen out peo­ple that answer ran­dom­ly. We also set a cri­te­ri­on for behav­iour­al task per­for­mance to make sure that peo­ple were pay­ing atten­tion to the task.

More gen­er­al­ly, we incen­tivized peo­ple by pay­ing them bonus money accord­ing to their task per­for­mance. I sus­pect that incen­tiviza­tion is the most cru­cial fac­tor to ensure engagement.

What are the main ways peo­ple mis­un­der­stand your thesis?

In our recent study, we show that peo­ple with rad­i­cal polit­i­cal beliefs show reduced insight into the cor­rect­ness of their deci­sions. This effect is present on both extremes of the polit­i­cal spec­trum. When I present those results, peo­ple often equate “rad­i­cal” with “far right” (and ignore the far left) which is not what we find. There might be some selec­tive lis­ten­ing going on when peo­ple inter­pret our findings!

What real-world prob­lem do you see that your research could impact?

In my eyes, under­stand­ing soci­etal polar­iza­tion is a cru­cial goal and there­fore I believe my research has direct rel­e­vance for an impor­tant issue of our time. Under­stand­ing the cog­ni­tive under­pin­nings of rad­i­cal beliefs will hope­ful­ly help us to coun­ter­act the process of rad­i­cal­iza­tion in the future.

“Online research might be a game chang­er in solv­ing the repli­ca­tion crisis.”

Has this study been published?

The study has been pub­lished in Cur­rent Biol­o­gy here.

How do you think online research is going to change your field?

Online research will have a huge impact on the field of psy­chol­o­gy and cog­ni­tive neu­ro­science. In par­tic­u­lar, online research might be a game chang­er in solv­ing the repli­ca­tion cri­sis. For instance, since it is rel­a­tive­ly easy, fast and inex­pen­sive to acquire online data, we have start­ed to run inter­nal repli­ca­tion sam­ples for all effects we find in our online stud­ies. This helps to estab­lish the robust­ness of our effects. Such an approach could move the field towards more repro­ducible findings.

What is the biggest advan­tage of online research methods?

The pos­si­bil­i­ty to acquire large data with­in a short time. The sam­ples also tend to be more rep­re­sen­ta­tive and diverse than the sam­ples we have usu­al­ly in the lab.

How did Gorilla make your life or research bet­ter, eas­i­er or faster?

For pilot­ing, it is great that the task builder pro­vides an inter­face with which it is pos­si­ble to cre­ate and test an exper­i­ment with­in a few hours/days. The abil­i­ty to con­duct com­plex behav­iour­al exper­i­ments online makes it pos­si­ble to acquire large data sets in very lit­tle time.

For you, what is the stand-out fea­ture in Gorilla?

The task builder is very help­ful for design­ing quick and easy behav­iour­al experiments.

What improve­ments would you like to see in Gorilla to make your research easier?

I think it would be great to have more inte­gra­tion between the task builder and the cod­ing inter­face. I might not be up to date on this, but my expe­ri­ence is that using the task builder is fast and easy but under­stand­ably restrict­ed in scope, where­as cod­ing a task from scratch takes a lot of time, but is very flex­i­ble. If it would be pos­si­ble to com­bine the best out of both worlds – for instance allow­ing edits to code cre­at­ed in the task builder – that would be extreme­ly helpful!

Response from Gorilla:

Hi Max,

We com­plete­ly agree! It’s already pos­si­ble to sup­ple­ment the task builder with script (see exam­ples here). But, we want to make this even bet­ter. We’re mak­ing some under­ly­ing archi­tec­ture changes to the task builder at the moment to make this possible.

Next year (2019), we expect there will be more options for users to add func­tion­al­i­ty to the task builder. In par­tic­u­lar, your two requests: (1) on the fly adap­tive trial gen­er­a­tion and (2) stim­uli (visu­al and audi­to­ry) gen­er­at­ed in script.

When you’re not work­ing, what do you enjoy doing?

I am a sports enthu­si­ast, espe­cial­ly enjoy­ing out­door sports like climb­ing and hiking.

What sci­ence book have you read recent­ly that you’d rec­om­mend to others?

Surf­ing Uncer­tain­ty by Andy Clark

Max Roll­wage
Cognitive Neuroscience: Decision-making, metacognition, computational modelling
PhD student
Wellcome Centre for Human Neuroimaging
Max Rollwage

Ready to get started?

More Spotlight Interviews

Mircea Zloteanu

Mircea Zloteanu

[get-spotlight-info] "We showed that not all “posed” emotions are created equal, and that research needs to consider this for us to have an accurate picture of human emotion recognition." Continue Reading Mircea Zloteanu