Some researchers have resisted the move to online research over the last few years, but the COVID crisis has forced many to switch to online methods. Taking research online brings many benefits, that may cause researchers to think twice before automatically reverting back to lab testing.
Benefits of online research
- Online data collection can be completed at an incredible speed. The tools for online research are now so good, that it can take only a few hours to create a study. Long gone are the days of painfully coding both the participant and server side experience. Couple this with any number of participant recruitment services and you can see the data come flying in.
- Go large scale and say goodbye to underpowered studies. As you no longer need to sit in the lab with each individual participant, multiple participants can complete your experiment simultaneously, leading to much larger samples. You can get data from thousands of participants in a day!
- Extend your reach and recruit the participants you need. Do you need a more diverse sample? Or a really specific group of participants? Integrate your experiment with a recruitment service like Prolific or SONA and reach groups that you couldn’t have done in the lab.
Benefits of lab research
So with all these benefits, why do we stay in the lab? Control! As researchers we like to feel like we’re in control in the lab. We may fear that remaining online removes this sense of control. It seems scary to have to trust that our participants will pay attention to the task we give them — especially if we’re not there to keep things on track. It’s scary to think about all the reasons why we may need to exclude participants, and to come up with a list of pre-defined exclusion criteria.
But in reality, these are things we should be thinking about anyway. Perhaps we don’t have full control in the lab after all — perhaps the control is just an illusion.
The illusion of control
When a participant comes into the lab we can interact with them and watch them complete the task. We can make sure they are in a quiet, distraction-free room, and sat at a sensible work space. Yet, we cannot control where their attention is focussed. They may look like they are paying attention to the task, but perhaps they are day dreaming or just not taking it seriously, and you can often only see this in the data later on in the research process.
Online we can ask participants to find a quiet space, but we can never be sure if they have done this. Again this is something that we wouldn’t necessarily spot until we look at the data.
But if this is something that is happening both in lab research and online research then we should be thinking about how we can deal with this issue. The best way to do this is through strong piloting of your study and working out objective exclusion criteria based on data quality. From this we can pre-register our criteria strengthening the trust other scientists can have in our work.
Pre-registering elements of our study like this is something that does actually give us some control over our research – thinking about these things ahead of data collection and analysis is incredibly important. More insight into maintaining data quality when you can’t see your participants can be found in Jenni Rodd’s BeOnline 2020 lecture.
By loading the video, you agree to YouTube’s privacy policy.
Learn more
The pain of face-to-face testing
Many types of behavioural science research involves working with one participant at a time, and bringing them to the lab. Maybe you can book 2 participants in per day, so to get a sample of 100 participants, that will be 50 days – but that’s only if every person turns up. Add in weekends and no-shows, you’re looking at around 2 months of data collection.
Instead imagine putting your study online and collecting data from 500 participants in one hour. Even if you had to exclude say 10% due to poor data quality, that’s still 450 participants in one hour. The amount of time and stress saved is immense! Many PhD students are funded using public funds, and so this time saving is also a cost saving and allows PhDs to focus on better experiment design or on task that will benefit their future research objectives.
The flexibility embedded in online research also allows for a more representative sample. Often face-to-face lab research will be missing out on participants who are unable to attend the lab during the working day. Going online allows people to complete your study at a time that suits them, meaning you can get reach participants that otherwise will not have been accounted for.
Unnatural behaviour
From another perspective, maybe too much control over participants is a bad thing – we put participants in an artificial situation, one that may be very new to them, and then sit and watch them complete a task. This may mean we are no longer getting a measure of ‘natural’ human behaviour, but how they respond in different circumstances.
A participant completing a task online cannot have their behaviour altered by our presence in the same way it could in the lab. In fact, in real life we rarely do one task in isolation – we often need to focus on one thing in the midst of distractions, and therefore research completed by participants at home may actually be more reflective of a real-world situation.
Strong benefits are becoming more evident
As researchers we’re all looking forward to the time when we can go back onto campuses and into labs safely. Yet, the illusion of control in lab face-to-face testing is being shattered, and the strong benefits of online research are becoming more evident. Online research tools allow us to conduct research faster, at larger scale and with greater reach which in turn gives us greater confidence in our results, and it’s here to stay.
Not already online? Why not?! We offer a best practice guide to online research as well as weekly onboarding webinars, so that researchers can hit the ground running. See you there!