How to Maintain Data Quality When You Can’t See Your Participants
Tra­di­tion­al­ly, most behav­iour­al research is done in lab­o­ra­to­ry set­tings. However, in recent years, the pop­u­lar­i­ty of online research has been growing, with more and more depart­ments and labs cre­at­ing and testing using online platforms.

Researchers are often nervous about col­lect­ing exper­i­men­tal data online because unlike lab-based exper­i­ments they cannot meet their par­tic­i­pants and can’t direct­ly observe their behaviour.

BeOnline2020 — Ensur­ing Data Quality Online

In her talk at the BeOn­line con­fer­ence this year, Jenni gave a great talk all about data quality, with many tips and hints. She sum­maris­es the issues that arise from ‘invis­i­ble’ par­tic­i­pants, sug­gest­ing a frame­work that helps researchers to (i) main­tain data quality and (ii) allows prin­ci­pled deci­sion about when online data col­lec­tion is (and is not) appro­pri­ate. She addi­tion­al­ly high­lights that this prin­ci­pled approach to data quality control is often missing in lab-based set­tings and that the lessons we have learned due to running exper­i­ments online can be broadly applied to a range of exper­i­men­tal approaches.

Article for the Asso­ci­a­tion for Psy­cho­log­i­cal Science — How to Main­tain Data Quality When You Can’t See Your Participants

For more details about main­tain­ing data quality, Jenni also wrote an article, which you can access here.

In the article, Jenni guides you through a five-stage process she pro­pos­es to go through prior to col­lect­ing data in any spe­cif­ic online experiment.


5‑stage process to help ensure data quality:

  1. Specify your data quality concerns
  2. Specify the worst case scenario
  3. Add new within-experiment safeguards
  4. Design experiment-spe­cif­ic exclu­sion criteria
  5. Pre-reg­is­ter your exclu­sion criteria