This inter­view was written in Decem­ber 2018 about the work Eva Poort did during her PhD at Uni­ver­si­ty College London.

 

What do you work on?

My area of research con­cerns the bilin­gual mental lexicon — the mental dic­tio­nary of people who speak more than one language.

Specif­i­cal­ly, I’m inter­est­ed to see how bilin­guals process words that exist in mul­ti­ple lan­guages, which can have either the same meaning, like the word “wolf” in Dutch and English, or a dif­fer­ent meaning, like the word “angel”, which in Dutch means “insect’s sting”.

For my research I mainly use lexical deci­sion tasks (Is this an exist­ing word or not?) and seman­tic relat­ed­ness tasks (Are these two words related to each other in meaning?).

“This has the poten­tial to really change how we think about how bilin­guals process words that exist in mul­ti­ple languages.”

What did you do using Gorilla?

I’ve by now run mul­ti­ple studies on Gorilla. In the first one, I used a cross-lingual long-term priming par­a­digm. In the first task of the experiment, par­tic­i­pants read sen­tences in Dutch that con­tained either a cognate, like “wolf”, or an inter­lin­gual homo­graph, like “angel”. Just to make sure that the par­tic­i­pants were actu­al­ly reading the sen­tences, I asked them to indi­cate for each sen­tence whether a sub­se­quent probe was related in meaning to the whole sentence.

Later on during the experiment, they com­plet­ed an English lexical deci­sion task which includ­ed some of the same cog­nates and inter­lin­gual homo­graphs as the first task. During this task, they were asked to indi­cate whether each stim­u­lus was a real English word or not.

In the past (see Poort, Warren, & Rodd, 2016), I had found that par­tic­i­pants respond­ed more quickly to cog­nates they had seen earlier on during the experiment, despite the fact that when they had seen them earlier it was in a Dutch context. In con­trast, having seen an inter­lin­gual homo­graph in a Dutch sen­tence context slowed the par­tic­i­pants down in the English task. Unfor­tu­nate­ly, the Gorilla experiment didn’t repli­cate this effect.

With my latest experiment, however, I showed that the type of task you use influ­ences the size of two well-known effects in the bilin­gual lit­er­a­ture, the cognate facil­i­ta­tion effect and the inter­lin­gual homo­graph inhi­bi­tion effect. In lexical deci­sion tasks, bilin­guals often process cog­nates more quickly than words that exist in only one of the lan­guages they know (e.g. “carrot”, which only exists in English and not in Dutch), but they process inter­lin­gual homo­graphs about as quickly those control words. In my experiment I used a seman­tic relat­ed­ness task, in which par­tic­i­pants saw pairs of words (e.g. “carrot” and “veg­etable”) and were asked to indi­cate whether those words were related in meaning or not.

In this task, the par­tic­i­pants did not respond more quickly (or slowly) to the pairs that includ­ed a cognate than to pairs that did not, but they did respond more slowly to pairs that includ­ed an inter­lin­gual homo­graph. This dif­fer­ence in the size of these effects in lexical deci­sion versus seman­tic relat­ed­ness has impor­tant impli­ca­tions for current the­o­ries of the bilin­gual mental lexicon.

What was your study protocol?

My exper­i­ments are always very complicated!

I’ll just describe the general pro­to­col briefly. I use a Ques­tion­naire for my demo­graph­ics ques­tions, then use Branch­es to filter out par­tic­i­pants who don’t meet my require­ments. I usually create my own tasks (I created both the lexical deci­sion task and seman­tic relat­ed­ness task described above) but have also used the in-built Towers of Hanoi task (with the maximum dura­tion feature). I also often use (chained) ran­domis­ers, but not really any of the other flow con­trollers (e.g. Coun­ter­bal­ance, Delay).

Did you include any special fea­tures in your study to ensure good quality data? If so, what did you do?

I usually do a com­bi­na­tion of the following:

• Ask par­tic­i­pants to fill in my demo­graph­ics ques­tion­naire at the start, so that I can use branch­es to filter out par­tic­i­pants who do not meet the (stated) par­tic­i­pa­tion requirements.
• Include some vocab­u­lary mea­sures and exclude par­tic­i­pants who do not meet some minimum score.
• Exclude par­tic­i­pants who do not achieve at least 80% correct on the main tasks of interest.
• Exclud­ed par­tic­i­pants who took too long to com­plete priming exper­i­ments, to make sure the priming delay is about the same for each participant.

 

(For the avoid­ance of doubt, par­tic­i­pants are always paid, but I exclude their data to ensure good data quality.)

“If I were start­ing my online research journey, I would def­i­nite­ly choose Gorilla.”

Has this study been published?

The study that used the cross-lingual long-term priming par­a­digm is Experiment 2 of this preprint:

https://psyarxiv.com/ert8k/

The seman­tic relat­ed­ness experiment is avail­able as a preprint here:

https://psyarxiv.com/fe2k8

For you, what is the stand-out feature in Gorilla?

There are too many! My exper­i­ments often consist of dif­fer­ent ver­sions with par­tic­i­pants com­plet­ing only one of those ver­sions, so the good ran­domi­sa­tion func­tion­al­i­ty and spread­sheet manip­u­la­tion feature have cer­tain­ly saved my life. And I haven’t used them yet, but I really like the sound of the Quota nodes. That will def­i­nite­ly make it easier in future to recruit equal numbers of par­tic­i­pants for the dif­fer­ent ver­sions in a multi-version experiment.

I’d also like to point out, although it isn’t a feature of Gorilla per se, the support offered by the team is phe­nom­e­nal. When­ev­er I get stuck with any­thing, they’re more than happy to help and they’re always open to feature requests (I’m so excited to use the Quota nodes!).

What is the most excit­ing piece of work or research you’ve ever done?

I would say the seman­tic relat­ed­ness experiment that I described above. The results allowed me to draw a par­al­lel with research con­duct­ed in the mono­lin­gual field and this has the poten­tial to really change how we think about how bilin­guals process words that exist in mul­ti­ple lan­guages (i.e. cog­nates and inter­lin­gual homographs).

How do you think online research is going to change your field?

Aside from the obvious advan­tages of being able to recruit greater numbers of par­tic­i­pants, I think for my field specif­i­cal­ly online research will make it much easier to study effects of lan­guage pro­fi­cien­cy and lan­guage dominance.

What is the biggest advan­tage of online research methods?

Being able to reach more diverse pop­u­la­tions and greater numbers of participants!

Why did you choose to use Gorilla?

For the very prac­ti­cal reason that UCL had just acquired a license for it and the soft­ware that I had been using (the Qualtrics Reac­tion Time Engine) no longer worked for me. That said, if I were start­ing my online research journey, I would def­i­nite­ly choose Gorilla, mainly for the ease of use and the wide range of features.

How did Gorilla make your life or research better, easier or faster?

It’s incred­i­bly easy and quick to set up an experiment in Gorilla! My first attempt at running an experiment online took me three months to set up and involved writing and editing a lot of com­pli­cat­ed JavaScript. In com­par­i­son, it took me about a month to set up a similar experiment in Gorilla. All the dif­fer­ent Nodes and Questionnaire/Task options also add so much func­tion­al­i­ty that it’s really pos­si­ble to run even the most com­pli­cat­ed experiment without too much hassle.

What improve­ments would you like to see in Gorilla to make your research easier?

It’s been a while since I last used Gorilla, but back then even though I could down­load the data for all nodes of the same task in one go, it would still give me a dif­fer­ent datafile for each node. My life would be even easier if these were merged auto­mat­i­cal­ly into a single file.

What do you believe to be true that you cannot prove (yet)?

I still (perhaps fool­ish­ly) believe that the cross-lingual priming effect is real, even though in follow-up exper­i­ments I haven’t been able to con­vinc­ing­ly repli­cate it.

Are there any online courses, pod­casts, dis­cus­sion groups or resources that you’d rec­om­mend to others?

Yes! Anyone doing any­thing with sta­tis­tics should do Daniël Lakens’ Improv­ing Your Sta­tis­ti­cal Infer­ences course on Cours­era. And learn how to use R.

When you’re not working, what do you enjoy doing?

I’ve got too many hobbies, really. I love to read, watch TV, knit, sew, go for long hikes, cook.

What’s your favourite science inter­net meme?

I don’t really have a favourite meme, but I really like the xkcd comics (and the related What If? website).