Let­ting go of the illu­sion of control

Some researchers have resist­ed the move to online research over the last few years, but the COVID cri­sis has forced many to switch to online meth­ods. Tak­ing research online brings many ben­e­fits, that may cause researchers to think twice before auto­mat­i­cal­ly revert­ing back to lab testing.

Ben­e­fits of online research

  • Online data col­lec­tion can be com­plet­ed at an incred­i­ble speed. The tools for online research are now so good, that it can take only a few hours to cre­ate a study. Long gone are the days of painful­ly cod­ing both the par­tic­i­pant and serv­er side expe­ri­ence. Cou­ple this with any num­ber of par­tic­i­pant recruit­ment ser­vices and you can see the data come fly­ing in.
  • Go large scale and say good­bye to under­pow­ered stud­ies. As you no longer need to sit in the lab with each indi­vid­ual par­tic­i­pant, mul­ti­ple par­tic­i­pants can com­plete your exper­i­ment simul­ta­ne­ous­ly, lead­ing to much larg­er sam­ples. You can get data from thou­sands of par­tic­i­pants in a day!
  • Extend your reach and recruit the par­tic­i­pants you need. Do you need a more diverse sam­ple? Or a real­ly spe­cif­ic group of par­tic­i­pants? Inte­grate your exper­i­ment with a recruit­ment ser­vice like Pro­lif­ic or SONA and reach groups that you couldn’t have done in the lab.

Ben­e­fits of lab research

So with all these ben­e­fits, why do we stay in the lab? Con­trol! As researchers we like to feel like we’re in con­trol in the lab. We may fear that remain­ing online removes this sense of con­trol. It seems scary to have to trust that our par­tic­i­pants will pay atten­tion to the task we give them — espe­cial­ly if we’re not there to keep things on track. It’s scary to think about all the rea­sons why we may need to exclude par­tic­i­pants, and to come up with a list of pre-defined exclu­sion criteria.

But in real­i­ty, these are things we should be think­ing about any­way. Per­haps we don’t have full con­trol in the lab after all — per­haps the con­trol is just an illusion.

The illu­sion of control

When a par­tic­i­pant comes into the lab we can inter­act with them and watch them com­plete the task. We can make sure they are in a quiet, dis­trac­tion-free room, and sat at a sen­si­ble work space. Yet, we can­not con­trol where their atten­tion is focussed. They may look like they are pay­ing atten­tion to the task, but per­haps they are day dream­ing or just not tak­ing it seri­ous­ly, and you can often only see this in the data later on in the research process.

Online we can ask par­tic­i­pants to find a quiet space, but we can never be sure if they have done this. Again this is some­thing that we wouldn’t nec­es­sar­i­ly spot until we look at the data.

But if this is some­thing that is hap­pen­ing both in lab research and online research then we should be think­ing about how we can deal with this issue. The best way to do this is through strong pilot­ing of your study and work­ing out objec­tive exclu­sion cri­te­ria based on data qual­i­ty. From this we can pre-reg­is­ter our cri­te­ria strength­en­ing the trust other sci­en­tists can have in our work.

Pre-reg­is­ter­ing ele­ments of our study like this is some­thing that does actu­al­ly give us some con­trol over our research – think­ing about these things ahead of data col­lec­tion and analy­sis is incred­i­bly impor­tant. More insight into main­tain­ing data qual­i­ty when you can’t see your par­tic­i­pants can be found in Jenni Rodd’s BeOn­line 2020 lec­ture.

YouTube

By load­ing the video, you agree to YouTube’s pri­va­cy pol­i­cy.
Learn more

Load video

The pain of face-to-face testing

Many types of behav­iour­al sci­ence research involves work­ing with one par­tic­i­pant at a time, and bring­ing them to the lab. Maybe you can book 2 par­tic­i­pants in per day, so to get a sam­ple of 100 par­tic­i­pants, that will be 50 days – but that’s only if every per­son turns up. Add in week­ends and no-shows, you’re look­ing at around 2 months of data collection.

Instead imag­ine putting your study online and col­lect­ing data from 500 par­tic­i­pants in one hour. Even if you had to exclude say 10% due to poor data qual­i­ty, that’s still 450 par­tic­i­pants in one hour. The amount of time and stress saved is immense! Many PhD stu­dents are fund­ed using pub­lic funds, and so this time sav­ing is also a cost sav­ing and allows PhDs to focus on bet­ter exper­i­ment design or on task that will ben­e­fit their future research objectives.

The flex­i­bil­i­ty embed­ded in online research also allows for a more rep­re­sen­ta­tive sam­ple. Often face-to-face lab research will be miss­ing out on par­tic­i­pants who are unable to attend the lab dur­ing the work­ing day. Going online allows peo­ple to com­plete your study at a time that suits them, mean­ing you can get reach par­tic­i­pants that oth­er­wise will not have been account­ed for.

Unnat­ur­al behaviour

From anoth­er per­spec­tive, maybe too much con­trol over par­tic­i­pants is a bad thing – we put par­tic­i­pants in an arti­fi­cial sit­u­a­tion, one that may be very new to them, and then sit and watch them com­plete a task. This may mean we are no longer get­ting a mea­sure of ‘nat­ur­al’ human behav­iour, but how they respond in dif­fer­ent circumstances.

A par­tic­i­pant com­plet­ing a task online can­not have their behav­iour altered by our pres­ence in the same way it could in the lab. In fact, in real life we rarely do one task in iso­la­tion – we often need to focus on one thing in the midst of dis­trac­tions, and there­fore research com­plet­ed by par­tic­i­pants at home may actu­al­ly be more reflec­tive of a real-world situation.

Strong ben­e­fits are becom­ing more evident

As researchers we’re all look­ing for­ward to the time when we can go back onto cam­pus­es and into labs safe­ly. Yet, the illu­sion of con­trol in lab face-to-face test­ing is being shat­tered, and the strong ben­e­fits of online research are becom­ing more evi­dent. Online research tools allow us to con­duct research faster, at larg­er scale and with greater reach which in turn gives us greater con­fi­dence in our results, and it’s here to stay.

Not already online? Why not?! We offer a best prac­tice guide to online research as well as week­ly onboard­ing webi­na­rs, so that researchers can hit the ground run­ning. See you there!