Eva Poort
December 2019

What do you work on?

My area of research con­cerns the bilin­gual men­tal lex­i­con — the men­tal dic­tio­nary of peo­ple who speak more than one language.

Specif­i­cal­ly, I’m inter­est­ed to see how bilin­guals process words that exist in mul­ti­ple lan­guages, which can have either the same mean­ing, like the word “wolf” in Dutch and Eng­lish, or a dif­fer­ent mean­ing, like the word “angel”, which in Dutch means “insect’s sting”.

For my research I main­ly use lex­i­cal deci­sion tasks (Is this an exist­ing word or not?) and seman­tic relat­ed­ness tasks (Are these two words relat­ed to each other in mean­ing?).

“This has the poten­tial to real­ly change how we think about how bilin­guals process words that exist in mul­ti­ple languages.”

What did you do using Gorilla?

I’ve by now run mul­ti­ple stud­ies on Gorilla. In the first one, I used a cross-lin­gual long-term prim­ing par­a­digm. In the first task of the exper­i­ment, par­tic­i­pants read sen­tences in Dutch that con­tained either a cog­nate, like “wolf”, or an inter­lin­gual homo­graph, like “angel”. Just to make sure that the par­tic­i­pants were actu­al­ly read­ing the sen­tences, I asked them to indi­cate for each sen­tence whether a sub­se­quent probe was relat­ed in mean­ing to the whole sentence.

Later on dur­ing the exper­i­ment, they com­plet­ed an Eng­lish lex­i­cal deci­sion task which includ­ed some of the same cog­nates and inter­lin­gual homo­graphs as the first task. Dur­ing this task, they were asked to indi­cate whether each stim­u­lus was a real Eng­lish word or not.

In the past (see Poort, War­ren, & Rodd, 2016), I had found that par­tic­i­pants respond­ed more quick­ly to cog­nates they had seen ear­li­er on dur­ing the exper­i­ment, despite the fact that when they had seen them ear­li­er it was in a Dutch con­text. In con­trast, hav­ing seen an inter­lin­gual homo­graph in a Dutch sen­tence con­text slowed the par­tic­i­pants down in the Eng­lish task. Unfor­tu­nate­ly, the Gorilla exper­i­ment didn’t repli­cate this effect.

With my lat­est exper­i­ment, how­ev­er, I showed that the type of task you use influ­ences the size of two well-known effects in the bilin­gual lit­er­a­ture, the cog­nate facil­i­ta­tion effect and the inter­lin­gual homo­graph inhi­bi­tion effect. In lex­i­cal deci­sion tasks, bilin­guals often process cog­nates more quick­ly than words that exist in only one of the lan­guages they know (e.g. “car­rot”, which only exists in Eng­lish and not in Dutch), but they process inter­lin­gual homo­graphs about as quick­ly those con­trol words. In my exper­i­ment I used a seman­tic relat­ed­ness task, in which par­tic­i­pants saw pairs of words (e.g. “car­rot” and “veg­etable”) and were asked to indi­cate whether those words were relat­ed in mean­ing or not.

In this task, the par­tic­i­pants did not respond more quick­ly (or slow­ly) to the pairs that includ­ed a cog­nate than to pairs that did not, but they did respond more slow­ly to pairs that includ­ed an inter­lin­gual homo­graph. This dif­fer­ence in the size of these effects in lex­i­cal deci­sion ver­sus seman­tic relat­ed­ness has impor­tant impli­ca­tions for cur­rent the­o­ries of the bilin­gual men­tal lexicon.

What was your study protocol?

My exper­i­ments are always very complicated!

I’ll just describe the gen­er­al pro­to­col briefly. I use a Ques­tion­naire for my demo­graph­ics ques­tions, then use Branch­es to fil­ter out par­tic­i­pants who don’t meet my require­ments. I usu­al­ly cre­ate my own tasks (I cre­at­ed both the lex­i­cal deci­sion task and seman­tic relat­ed­ness task described above) but have also used the in-built Tow­ers of Hanoi task (with the max­i­mum dura­tion fea­ture). I also often use (chained) ran­domis­ers, but not real­ly any of the other flow con­trollers (e.g. Coun­ter­bal­ance, Delay).

Did you include any spe­cial fea­tures in your study to ensure good qual­i­ty data? If so, what did you do?

I usu­al­ly do a com­bi­na­tion of the following:

  • Ask par­tic­i­pants to fill in my demo­graph­ics ques­tion­naire at the start, so that I can use branch­es to fil­ter out par­tic­i­pants who do not meet the (stat­ed) par­tic­i­pa­tion requirements.
  • Include some vocab­u­lary mea­sures and exclude par­tic­i­pants who do not meet some min­i­mum score.
  • Exclude par­tic­i­pants who do not achieve at least 80% cor­rect on the main tasks of interest.
  • Exclud­ed par­tic­i­pants who took too long to com­plete prim­ing exper­i­ments, to make sure the prim­ing delay is about the same for each participant.


(For the avoid­ance of doubt, par­tic­i­pants are always paid, but I exclude their data to ensure good data quality.)

“If I were start­ing my online research jour­ney, I would def­i­nite­ly choose Gorilla.”

Has this study been published?

The study that used the cross-lin­gual long-term prim­ing par­a­digm is Exper­i­ment 2 of this preprint:


The seman­tic relat­ed­ness exper­i­ment is avail­able as a preprint here:


For you, what is the stand-out fea­ture in Gorilla?

There are too many! My exper­i­ments often con­sist of dif­fer­ent ver­sions with par­tic­i­pants com­plet­ing only one of those ver­sions, so the good ran­domi­sa­tion func­tion­al­i­ty and spread­sheet manip­u­la­tion fea­ture have cer­tain­ly saved my life. And I haven’t used them yet, but I real­ly like the sound of the Quota nodes. That will def­i­nite­ly make it eas­i­er in future to recruit equal num­bers of par­tic­i­pants for the dif­fer­ent ver­sions in a multi-ver­sion experiment.

I’d also like to point out, although it isn’t a fea­ture of Gorilla per se, the sup­port offered by the team is phe­nom­e­nal. When­ev­er I get stuck with any­thing, they’re more than happy to help and they’re always open to fea­ture requests (I’m so excit­ed to use the Quota nodes!).

What is the most excit­ing piece of work or research you’ve ever done?

I would say the seman­tic relat­ed­ness exper­i­ment that I described above. The results allowed me to draw a par­al­lel with research con­duct­ed in the mono­lin­gual field and this has the poten­tial to real­ly change how we think about how bilin­guals process words that exist in mul­ti­ple lan­guages (i.e. cog­nates and inter­lin­gual homographs).

How do you think online research is going to change your field?

Aside from the obvi­ous advan­tages of being able to recruit greater num­bers of par­tic­i­pants, I think for my field specif­i­cal­ly online research will make it much eas­i­er to study effects of lan­guage pro­fi­cien­cy and lan­guage dominance.

What is the biggest advan­tage of online research methods?

Being able to reach more diverse pop­u­la­tions and greater num­bers of participants!

Why did you choose to use Gorilla?

For the very prac­ti­cal rea­son that UCL had just acquired a license for it and the soft­ware that I had been using (the Qualtrics Reac­tion Time Engine) no longer worked for me. That said, if I were start­ing my online research jour­ney, I would def­i­nite­ly choose Gorilla, main­ly for the ease of use and the wide range of features.

How did Gorilla make your life or research bet­ter, eas­i­er or faster?

It’s incred­i­bly easy and quick to set up an exper­i­ment in Gorilla! My first attempt at run­ning an exper­i­ment online took me three months to set up and involved writ­ing and edit­ing a lot of com­pli­cat­ed JavaScript. In com­par­i­son, it took me about a month to set up a sim­i­lar exper­i­ment in Gorilla. All the dif­fer­ent Nodes and Questionnaire/Task options also add so much func­tion­al­i­ty that it’s real­ly pos­si­ble to run even the most com­pli­cat­ed exper­i­ment with­out too much hassle.

What improve­ments would you like to see in Gorilla to make your research easier?

It’s been a while since I last used Gorilla, but back then even though I could down­load the data for all nodes of the same task in one go, it would still give me a dif­fer­ent datafile for each node. My life would be even eas­i­er if these were merged auto­mat­i­cal­ly into a sin­gle file.

What do you believe to be true that you can­not prove (yet)?

I still (per­haps fool­ish­ly) believe that the cross-lin­gual prim­ing effect is real, even though in fol­low-up exper­i­ments I haven’t been able to con­vinc­ing­ly repli­cate it.

Are there any online cours­es, pod­casts, dis­cus­sion groups or resources that you’d rec­om­mend to others?

Yes! Any­one doing any­thing with sta­tis­tics should do Daniël Lak­ens’ Improv­ing Your Sta­tis­ti­cal Infer­ences course on Cours­era. And learn how to use R.

When you’re not work­ing, what do you enjoy doing?

I’ve got too many hob­bies, real­ly. I love to read, watch TV, knit, sew, go for long hikes, cook.

What’s your favourite sci­ence inter­net meme?

I don’t real­ly have a favourite meme, but I real­ly like the xkcd comics (and the relat­ed What If? web­site).

Eva Poort
PhD student (post-viva)
University College London
Eva Poort

Ready to get started?

More Spotlight Interviews

Anqi Hu

Anqi Hu

[get-spotlight-info] “Online research has the great potential for us to reach out to a more representative sample and to create a more participant-friendly research experience. It is tremendously helpful for studies that investigate individual differences and learning in children.” Continue Reading Anqi Hu