Xiao Hu
May 2020

What do you work on?

My study focuses on the role of metacog­ni­tion in cog­ni­tive offloading.

Cog­ni­tive offload­ing refers to our use of exter­nal envi­ron­ment to reduce our mental demand in daily tasks. For example, we write notes on paper or smart­phones in order not to forget shop­ping lists or upcom­ing appointments.

I am inter­est­ed in whether our deci­sion to offload infor­ma­tion is related to our eval­u­a­tions of ability on the task – a form of metacognition.

What did you do using Gorilla?

We inves­ti­gat­ed the role of metacog­ni­tive eval­u­a­tion on cog­ni­tive offload­ing in a memory task. Our hypoth­e­sis was that metacog­ni­tion should affect our offload­ing deci­sions during both memory encod­ing and retrieval. Specif­i­cal­ly, we hypoth­e­sized that:

(a) people would choose to save studied items when they believed it was dif­fi­cult to recall the items in future;

(b) they would ask for help from exter­nal sources during retrieval when they were not con­fi­dent about their inter­nal memory.

We used Gorilla to collect data in two exper­i­ments of our paper (Exper­i­ments 2b and 3). We asked par­tic­i­pants to learn word pairs, and they could choose to save some pairs into com­put­er. In a later memory test, they could decide whether to ask for help from the com­put­er. They would receive hints at a small cost if they chose to ask for help for saved pairs. Before the test for each pair, par­tic­i­pants also rated their con­fi­dence about recall­ing the pair.

“Con­fi­dence about memory (rather than memory per­for­mance itself) is a key factor in driving offload­ing decisions.”

What did you do find?

We found support for both of our hypothe­ses, and accom­mo­dat­ed our find­ings by devel­op­ing a Bayesian com­pu­ta­tion­al model in which par­tic­i­pants’ eval­u­a­tion of inter­nal memory ability was neg­a­tive­ly cor­re­lat­ed with their beliefs about the per­for­mance boost gained from hints.

Togeth­er our find­ings high­light a close link between metacog­ni­tion and cog­ni­tive offload­ing, sug­gest­ing that con­fi­dence about memory (rather than memory per­for­mance itself) is a key factor in driving offload­ing decisions.

What real-world problem do you see that your research could impact?

The use of cog­ni­tive offload­ing during learn­ing is increas­ing­ly ubiq­ui­tous due to the rapid devel­op­ment of tech­nol­o­gy. By under­stand­ing when and how people rely on these exter­nal tools during learn­ing, psy­chol­o­gists and edu­ca­tors can guide people towards more effec­tive use of modern tech­nolo­gies to max­imise their benefit for memory performance.

What was your study protocol?

In the experiment, I first used two ques­tion­naires to collect consent forms and demo­graph­ic data. Then I used a task to check whether par­tic­i­pants entered fullscreen mode. Next par­tic­i­pants per­formed six tasks to prac­tice the encod­ing and retrieval phase in the experiment.

Before the formal experiment, I used a Ran­domiser to ran­dom­ly allo­cate par­tic­i­pants into two con­di­tions because font color in the con­fi­dence rating task was coun­ter­bal­anced across par­tic­i­pants. Finally, par­tic­i­pants fin­ished the formal experiment, and I used a Repeat node to repeat the exper­i­men­tal pro­ce­dure in three blocks.

Did you include any special fea­tures in your study to ensure good quality data? If so, what did you do?

To make sure par­tic­i­pants under­stood the exper­i­men­tal pro­ce­dure, we divided the prac­tice stage into several short blocks and asked par­tic­i­pants to sep­a­rate­ly prac­tice each phase of the task.

We also set exclu­sion cri­te­ria and planned to remove par­tic­i­pants with very low memory per­for­mance or short response times (for­tu­nate­ly all of our par­tic­i­pants passed these exclu­sion criteria).

In addi­tion, we incen­tivized people to engage with the task by paying them bonus money based on their per­for­mance in memory test.

Has this study been published?

The study has been pub­lished in Cog­ni­tion. You can find it here.

“I was able to collect pilot data online in only a few hours and then make revi­sions on the tasks based on these results within one or two days.”

What is the most excit­ing piece of work or research you’ve ever done?

I am most excited about the study I describe here. I carried out this work under the super­vi­sion of Dr. Steve Fleming when I was a vis­it­ing student at the Well­come Centre for Human Neu­roimag­ing, Uni­ver­si­ty College London. It was a really great expe­ri­ence for me to work with Steve. He taught me how to combine com­pu­ta­tion­al mod­el­ling and behav­iour­al exper­i­ments when study­ing metacognition.

What is the biggest advan­tage of online research methods?

The ability to collect a large amount of data from a rep­re­sen­ta­tive par­tic­i­pant pool within a very short time.

How do you think online research is going to change your field?

With online research tools, I can work from home and yet still collect large amounts of data within a matter of hours. This will speed up the through­put and reli­a­bil­i­ty of psy­cho­log­i­cal science. It is also pos­si­ble to build online exper­i­ments and then use mobile devices (e.g. tablets) to collect data from a large number of par­tic­i­pants in a field study (such as testing chil­dren in class­room), which has impor­tant impli­ca­tions for devel­op­men­tal and edu­ca­tion­al psychology.

How did Gorilla make your life or research better, easier or faster?

In Gorilla, it was very con­ve­nient to clone pre­vi­ous exper­i­ments and then revise the exper­i­men­tal pro­ce­dure in task builder. I was able to collect pilot data online in only a few hours and then make revi­sions on the tasks based on these results within one or two days. The commit history was also useful for keeping track of what revi­sions I had made.

For you, what is the stand-out feature in Gorilla?

The task builder is very easy to under­stand and con­ve­nient to use. I can also add scripts to make the tasks more flex­i­ble (such as linking stimuli across tasks or enter­ing fullscreen mode).

What improve­ments would you like to see in Gorilla to make your research easier?

It would be great to provide intro­duc­to­ry tuto­ri­als (or links to rel­e­vant tuto­ri­als) on Javascript to allow advanced users to modify scripts from the exam­ples in task builder.

Response from Gorilla:

Hi Xiao,

We com­plete­ly agree! We have lots of example scripts here, and they are very well com­ment­ed, but they aren’t search­able. We also have some basic infor­ma­tion about the key func­tions here but it’s not complete.

We hope to write a more com­plete guide in the next few months.

Until then, the best thing to do is to contact the support desk and we can either point you in the right direc­tion or help out!

Who or what orig­i­nal­ly inspired you to work in your field of research?

My super­vi­sor, Dr. Liang Luo, who has guided me a lot about how to design and conduct exper­i­ments in the field of metamemory.

If you could inter­view any researcher (alive or dead), who would it be and why?

Dr. Asher Koriat, who has pro­posed a series of impor­tant the­o­ries about metamem­o­ry process, and is one of the great­est leaders in the field of metamemory.

Are there any online courses, pod­casts, dis­cus­sion groups or resources that you’d rec­om­mend to others?

I would like to rec­om­mend a course about Bayesian sta­tis­tics on Cousera named Bayesian Sta­tis­tics: From Concept to Data Analy­sis, which is a good intro­duc­tion for people who are inter­est­ed in Bayesian sta­tis­tics but not sure where to start.

When you’re not working, what do you enjoy doing?

I enjoy reading phi­los­o­phy and history books, and also Japan­ese comics (I am a big fan of One Piece).

What science book have you read recent­ly that you’d rec­om­mend to others?

Forty Studies That Changed Psy­chol­o­gy by Roger R. Hock. This book pro­vides a summary of impor­tant studies that have impact­ed the field of psychology.

Xiao Hu
Picture showing a test glass Memory, metacognition, computational modelling
Picture showing an university graduates hat PhD student
Picture showing a School Beijing Normal University
Xiao Hu

Ready to get started?

More Spotlight Interviews

Xiao Hu

Xiao Hu

[get-spotlight-info] "I am interested in whether our decision to offload information is related to our evaluations of ability on the task – a form of metacognition.” Continue Reading Xiao Hu

Alex Jones

Alex Jones

[get-spotlight-info] "People unconsciously associate someone who looks Extraverted with words describing that trait. Using Gorilla, we are now expanding these studies to explain the effect using clinical populations.” Continue Reading Alex Jones