How to Maintain Data Quality When You Can’t See Your Participants
Main­tain­ing Data Qual­i­ty When Con­duct­ing Exper­i­ments Online
Tra­di­tion­al­ly, most behav­iour­al research is done in lab­o­ra­to­ry set­tings. How­ev­er, in recent years, the pop­u­lar­i­ty of online research has been grow­ing, with more and more depart­ments and labs cre­at­ing and test­ing using online platforms.

Researchers are often ner­vous about col­lect­ing exper­i­men­tal data online because unlike lab-based exper­i­ments they can­not meet their par­tic­i­pants and can’t direct­ly observe their behaviour.

BeOnline2020 — Ensur­ing Data Qual­i­ty Online

In her talk at the BeOn­line con­fer­ence this year, Jenni gave a great talk all about data qual­i­ty, with many tips and hints. She sum­maris­es the issues that arise from ‘invis­i­ble’ par­tic­i­pants, sug­gest­ing a frame­work that helps researchers to (i) main­tain data qual­i­ty and (ii) allows prin­ci­pled deci­sion about when online data col­lec­tion is (and is not) appro­pri­ate. She addi­tion­al­ly high­lights that this prin­ci­pled approach to data qual­i­ty con­trol is often miss­ing in lab-based set­tings and that the lessons we have learned due to run­ning exper­i­ments online can be broad­ly applied to a range of exper­i­men­tal approaches.


By load­ing the video, you agree to YouTube’s pri­va­cy pol­i­cy.
Learn more

Load video

Arti­cle for the Asso­ci­a­tion for Psy­cho­log­i­cal Sci­ence — How to Main­tain Data Qual­i­ty When You Can’t See Your Participants

For more details about main­tain­ing data qual­i­ty, Jenni also wrote an arti­cle, which you can access here.

In the arti­cle, Jenni guides you through a five-stage process she pro­pos­es to go through prior to col­lect­ing data in any spe­cif­ic online experiment.


5‑stage process to help ensure data quality:

  1. Spec­i­fy your data qual­i­ty concerns
  2. Spec­i­fy the worst case scenario
  3. Add new with­in-exper­i­ment safeguards
  4. Design exper­i­ment-spe­cif­ic exclu­sion criteria
  5. Pre-reg­is­ter your exclu­sion criteria