What do you work on?

My study focuses on the role of metacog­ni­tion in cog­ni­tive offloading.

Cog­ni­tive offload­ing refers to our use of exter­nal envi­ron­ment to reduce our mental demand in daily tasks. For example, we write notes on paper or smart­phones in order not to forget shop­ping lists or upcom­ing appointments.

I am inter­est­ed in whether our deci­sion to offload infor­ma­tion is related to our eval­u­a­tions of ability on the task – a form of metacognition.

What did you do using Gorilla?

We inves­ti­gat­ed the role of metacog­ni­tive eval­u­a­tion on cog­ni­tive offload­ing in a memory task. Our hypoth­e­sis was that metacog­ni­tion should affect our offload­ing deci­sions during both memory encod­ing and retrieval. Specif­i­cal­ly, we hypoth­e­sized that:

(a) people would choose to save studied items when they believed it was dif­fi­cult to recall the items in future;

(b) they would ask for help from exter­nal sources during retrieval when they were not con­fi­dent about their inter­nal memory.

We used Gorilla to collect data in two exper­i­ments of our paper (Exper­i­ments 2b and 3). We asked par­tic­i­pants to learn word pairs, and they could choose to save some pairs into com­put­er. In a later memory test, they could decide whether to ask for help from the com­put­er. They would receive hints at a small cost if they chose to ask for help for saved pairs. Before the test for each pair, par­tic­i­pants also rated their con­fi­dence about recall­ing the pair.

“Con­fi­dence about memory (rather than memory per­for­mance itself) is a key factor in driving offload­ing decisions.”

What did you do find?

We found support for both of our hypothe­ses, and accom­mo­dat­ed our find­ings by devel­op­ing a Bayesian com­pu­ta­tion­al model in which par­tic­i­pants’ eval­u­a­tion of inter­nal memory ability was neg­a­tive­ly cor­re­lat­ed with their beliefs about the per­for­mance boost gained from hints.

Togeth­er our find­ings high­light a close link between metacog­ni­tion and cog­ni­tive offload­ing, sug­gest­ing that con­fi­dence about memory (rather than memory per­for­mance itself) is a key factor in driving offload­ing decisions.

What real-world problem do you see that your research could impact?

The use of cog­ni­tive offload­ing during learn­ing is increas­ing­ly ubiq­ui­tous due to the rapid devel­op­ment of tech­nol­o­gy. By under­stand­ing when and how people rely on these exter­nal tools during learn­ing, psy­chol­o­gists and edu­ca­tors can guide people towards more effec­tive use of modern tech­nolo­gies to max­imise their benefit for memory performance.

What was your study protocol?

In the experiment, I first used two ques­tion­naires to collect consent forms and demo­graph­ic data. Then I used a task to check whether par­tic­i­pants entered fullscreen mode. Next par­tic­i­pants per­formed six tasks to prac­tice the encod­ing and retrieval phase in the experiment.

Before the formal experiment, I used a Ran­domiser to ran­dom­ly allo­cate par­tic­i­pants into two con­di­tions because font color in the con­fi­dence rating task was coun­ter­bal­anced across par­tic­i­pants. Finally, par­tic­i­pants fin­ished the formal experiment, and I used a Repeat node to repeat the exper­i­men­tal pro­ce­dure in three blocks.

Did you include any special fea­tures in your study to ensure good quality data? If so, what did you do?

To make sure par­tic­i­pants under­stood the exper­i­men­tal pro­ce­dure, we divided the prac­tice stage into several short blocks and asked par­tic­i­pants to sep­a­rate­ly prac­tice each phase of the task.

We also set exclu­sion cri­te­ria and planned to remove par­tic­i­pants with very low memory per­for­mance or short response times (for­tu­nate­ly all of our par­tic­i­pants passed these exclu­sion criteria).

In addi­tion, we incen­tivized people to engage with the task by paying them bonus money based on their per­for­mance in memory test.

Has this study been published?

The study has been pub­lished in Cog­ni­tion. You can find it here.

“I was able to collect pilot data online in only a few hours and then make revi­sions on the tasks based on these results within one or two days.”

What is the most excit­ing piece of work or research you’ve ever done?

I am most excited about the study I describe here. I carried out this work under the super­vi­sion of Dr. Steve Fleming when I was a vis­it­ing student at the Well­come Centre for Human Neu­roimag­ing, Uni­ver­si­ty College London. It was a really great expe­ri­ence for me to work with Steve. He taught me how to combine com­pu­ta­tion­al mod­el­ling and behav­iour­al exper­i­ments when study­ing metacognition.

What is the biggest advan­tage of online research methods?

The ability to collect a large amount of data from a rep­re­sen­ta­tive par­tic­i­pant pool within a very short time.

How do you think online research is going to change your field?

With online research tools, I can work from home and yet still collect large amounts of data within a matter of hours. This will speed up the through­put and reli­a­bil­i­ty of psy­cho­log­i­cal science. It is also pos­si­ble to build online exper­i­ments and then use mobile devices (e.g. tablets) to collect data from a large number of par­tic­i­pants in a field study (such as testing chil­dren in class­room), which has impor­tant impli­ca­tions for devel­op­men­tal and edu­ca­tion­al psychology.

How did Gorilla make your life or research better, easier or faster?

In Gorilla, it was very con­ve­nient to clone pre­vi­ous exper­i­ments and then revise the exper­i­men­tal pro­ce­dure in task builder. I was able to collect pilot data online in only a few hours and then make revi­sions on the tasks based on these results within one or two days. The commit history was also useful for keeping track of what revi­sions I had made.

For you, what is the stand-out feature in Gorilla?

The task builder is very easy to under­stand and con­ve­nient to use. I can also add scripts to make the tasks more flex­i­ble (such as linking stimuli across tasks or enter­ing fullscreen mode).

What improve­ments would you like to see in Gorilla to make your research easier?

It would be great to provide intro­duc­to­ry tuto­ri­als (or links to rel­e­vant tuto­ri­als) on Javascript to allow advanced users to modify scripts from the exam­ples in task builder.

Response from Gorilla:

Hi Xiao,

We com­plete­ly agree! We have lots of example scripts here, and they are very well com­ment­ed, but they aren’t search­able. We also have some basic infor­ma­tion about the key func­tions here but it’s not complete.

We hope to write a more com­plete guide in the next few months.

Until then, the best thing to do is to contact the support desk and we can either point you in the right direc­tion or help out!

Who or what orig­i­nal­ly inspired you to work in your field of research?

My super­vi­sor, Dr. Liang Luo, who has guided me a lot about how to design and conduct exper­i­ments in the field of metamemory.

If you could inter­view any researcher (alive or dead), who would it be and why?

Dr. Asher Koriat, who has pro­posed a series of impor­tant the­o­ries about metamem­o­ry process, and is one of the great­est leaders in the field of metamemory.

Are there any online courses, pod­casts, dis­cus­sion groups or resources that you’d rec­om­mend to others?

I would like to rec­om­mend a course about Bayesian sta­tis­tics on Cousera named Bayesian Sta­tis­tics: From Concept to Data Analy­sis, which is a good intro­duc­tion for people who are inter­est­ed in Bayesian sta­tis­tics but not sure where to start.

When you’re not working, what do you enjoy doing?

I enjoy reading phi­los­o­phy and history books, and also Japan­ese comics (I am a big fan of One Piece).

What science book have you read recent­ly that you’d rec­om­mend to others?

Forty Studies That Changed Psy­chol­o­gy by Roger R. Hock. This book pro­vides a summary of impor­tant studies that have impact­ed the field of psychology.