Some researchers have resist­ed the move to online research over the last few years, but the COVID crisis has forced many to switch to online methods. Taking research online brings many ben­e­fits, that may cause researchers to think twice before auto­mat­i­cal­ly revert­ing back to lab testing.

Ben­e­fits of online research

• Online data col­lec­tion can be com­plet­ed at an incred­i­ble speed. The tools for online research are now so good, that it can take only a few hours to create a study. Long gone are the days of painful­ly coding both the par­tic­i­pant and server side expe­ri­ence. Couple this with any number of par­tic­i­pant recruit­ment ser­vices and you can see the data come flying in.
• Go large scale and say goodbye to under­pow­ered studies. As you no longer need to sit in the lab with each indi­vid­ual par­tic­i­pant, mul­ti­ple par­tic­i­pants can com­plete your experiment simul­ta­ne­ous­ly, leading to much larger samples. You can get data from thou­sands of par­tic­i­pants in a day!
• Extend your reach and recruit the par­tic­i­pants you need. Do you need a more diverse sample? Or a really spe­cif­ic group of par­tic­i­pants? Inte­grate your experiment with a recruit­ment service like Pro­lif­ic or SONA and reach groups that you couldn’t have done in the lab.

Ben­e­fits of lab research

So with all these ben­e­fits, why do we stay in the lab? Control! As researchers we like to feel like we’re in control in the lab. We may fear that remain­ing online removes this sense of control. It seems scary to have to trust that our par­tic­i­pants will pay atten­tion to the task we give them — espe­cial­ly if we’re not there to keep things on track. It’s scary to think about all the reasons why we may need to exclude par­tic­i­pants, and to come up with a list of pre-defined exclu­sion criteria.

But in reality, these are things we should be think­ing about anyway. Perhaps we don’t have full control in the lab after all — perhaps the control is just an illusion.

The illu­sion of control

When a par­tic­i­pant comes into the lab we can inter­act with them and watch them com­plete the task. We can make sure they are in a quiet, dis­trac­tion-free room, and sat at a sen­si­ble work space. Yet, we cannot control where their atten­tion is focussed. They may look like they are paying atten­tion to the task, but perhaps they are day dream­ing or just not taking it seri­ous­ly, and you can often only see this in the data later on in the research process.

Online we can ask par­tic­i­pants to find a quiet space, but we can never be sure if they have done this. Again this is some­thing that we wouldn’t nec­es­sar­i­ly spot until we look at the data.

But if this is some­thing that is hap­pen­ing both in lab research and online research then we should be think­ing about how we can deal with this issue. The best way to do this is through strong pilot­ing of your study and working out objec­tive exclu­sion cri­te­ria based on data quality. From this we can pre-reg­is­ter our cri­te­ria strength­en­ing the trust other sci­en­tists can have in our work.

Pre-reg­is­ter­ing ele­ments of our study like this is some­thing that does actu­al­ly give us some control over our research – think­ing about these things ahead of data col­lec­tion and analy­sis is incred­i­bly impor­tant. More insight into main­tain­ing data quality when you can’t see your par­tic­i­pants can be found in Jenni Rodd’s BeOn­line 2020 lecture.

The pain of face-to-face testing

Many types of behav­iour­al science research involves working with one par­tic­i­pant at a time, and bring­ing them to the lab. Maybe you can book 2 par­tic­i­pants in per day, so to get a sample of 100 par­tic­i­pants, that will be 50 days – but that’s only if every person turns up. Add in week­ends and no-shows, you’re looking at around 2 months of data collection.

Instead imagine putting your study online and col­lect­ing data from 500 par­tic­i­pants in one hour. Even if you had to exclude say 10% due to poor data quality, that’s still 450 par­tic­i­pants in one hour. The amount of time and stress saved is immense! Many PhD stu­dents are funded using public funds, and so this time saving is also a cost saving and allows PhDs to focus on better experiment design or on task that will benefit their future research objectives.

The flex­i­bil­i­ty embed­ded in online research also allows for a more rep­re­sen­ta­tive sample. Often face-to-face lab research will be missing out on par­tic­i­pants who are unable to attend the lab during the working day. Going online allows people to com­plete your study at a time that suits them, meaning you can get reach par­tic­i­pants that oth­er­wise will not have been account­ed for.

Unnat­ur­al behaviour

From another per­spec­tive, maybe too much control over par­tic­i­pants is a bad thing – we put par­tic­i­pants in an arti­fi­cial sit­u­a­tion, one that may be very new to them, and then sit and watch them com­plete a task. This may mean we are no longer getting a measure of ‘natural’ human behav­iour, but how they respond in dif­fer­ent circumstances.

A par­tic­i­pant com­plet­ing a task online cannot have their behav­iour altered by our pres­ence in the same way it could in the lab. In fact, in real life we rarely do one task in iso­la­tion – we often need to focus on one thing in the midst of dis­trac­tions, and there­fore research com­plet­ed by par­tic­i­pants at home may actu­al­ly be more reflec­tive of a real-world situation.

Strong ben­e­fits are becom­ing more evident

As researchers we’re all looking forward to the time when we can go back onto cam­pus­es and into labs safely. Yet, the illu­sion of control in lab face-to-face testing is being shat­tered, and the strong ben­e­fits of online research are becom­ing more evident. Online research tools allow us to conduct research faster, at larger scale and with greater reach which in turn gives us greater con­fi­dence in our results, and it’s here to stay.

Not already online? Why not?! We offer a best prac­tice guide to online research as well as weekly onboard­ing webi­na­rs, so that researchers can hit the ground running. See you there!