Support Home

Timing in Gorilla

  • Overview
  • Online Timing Accuracy
  • Browser-based timing
  • Displaying Stimuli
  • Measuring Reaction Times
  • Validating Timing Accuracy in Gorilla
  • Further Resources

Overview


In online research, the main sources of inaccuracy in regards to timing come from peripheral devices, which affect both native applications (i.e. installed on a computer in the lab) and browser-based ones alike. This paper gives a good overview of the issues involved in online research, and contains a detailed section on timing.

In Gorilla, we use high resolution timers for accurate reaction times and frame counting for accurate stimuli presentation times - techniques that have only recently become available in all major browsers.

Click through sections on the menu to the left to learn all about timing in Gorilla and make sure to check out further resources for more information on timing in online research!


Online Timing Accuracy

When conducting research online, rather than in the lab, it's important to understand how timing accuracy changes.
The schematic below shows the main differences:

Modern techniques refers to the Performance.now() function common in all major browsers which provides microsecond timing (read more about Performance.now() function here). We do not have control of the underlying operating system, so we cannot make the same OS-level timing calls that native software can. While clearly there will always be some experiments that require extremely high fidelity timing, the precision offered by Gorilla is appropriate for a wide range of research.

The latency between the cloud storage and local computer is irrelevant because Gorilla downloads all task information (i.e. stimuli) to the local machine in advance and collects local timestamps.

To talk through each element in the picture above:

  • Stimulus presentation is constrained by the refresh rate of the monitor which is typically 16.6ms. Browser displays can now be synchronised to the refresh rate (thank you online gamers!), so that you can present for a single frame or multiples thereof.
  • On most computers the keyboard and mouse will be polled every 8 ms. This varies a bit between devices.
  • If you are using a mobile device, touch screen are typically polled every ~20ms
  • The latency between the cloud storage and local computer is irrelevant because Gorilla downloads all task information (i.e. stimuli) to the local machine in advance and calculates the reaction time based on the local timestamps.

See the further resources section for more information on timing in online research.

Browser-based timing

Much of the criticism of browser-based timings stem from the inaccuracy of JavaScript's setTimeout(), setInterval() and Date.now() functions. Within the last few years, more accurate techniques have become available:

  • performance.now() offers microsecond-precision timestamps. This was first introduced in Chrome in 2012, and iOS was the last to adopt it in 2015.
  • requestAnimationFrame() allows us to run logic every time the screen is about to be refreshed, and should be synchronised to the refresh rate. This was first introduced in Firefox in 2012, and Android was the last to adopt it in 2013.

Both features are now standard in all major browsers. See more information about performance.now() here and about requestAnimationFrame() here.


Displaying Stimuli

The key issue with displaying stimuli is that we are fundamentally limited to the refresh rate of the display. Most modern displays refresh at 60Hz, which is every 16.67 milliseconds. This means that stimuli can only ever be displayed for multiples of 16.67 milliseconds - without specialist equipment, you cannot show a stimuli image for 40ms; you can either have 33.3ms (two frames) or 50ms (three frames).

This issue affects native programs and browser-based ones equally, as they are both ending up on the same display.

Gorilla uses requestAnimationFrame() to count frames and ensure that a stimuli is shown for as accurate a time as possible, by evaluating for each frame whether it would be more accurate to display for another frame (because there is more than 8ms left to display) or to stop on this frame (because there is less than 8ms left to show).


Measuring Reaction Times

Whenever a new screen is displayed in Gorilla, a timestamp is recorded using performance.now(). When a response is recorded (e.g. a keypress or a mouse click), a second timestamp is taken, and the reaction time is then the difference between the two.

Exactly how accurate the response is depends chiefly on the latency of the input device. The default USB polling rate in Windows is 120Hz, or every 8.3ms. Therefore, there could be up to an 8.3ms error on keyboard presses or mouse clicks for peripherals connected via USB.

Touchscreens have higher latency still. The fastest devices are now at around 20ms, and getting faster all the time.


Validating Timing Accuracy in Gorilla

In 2019, we published our paper on timing accuracy in Gorilla. You can read it here. We used a simplified flanker task taken from the attentional network task (Rueda, Posner, & Rothbart, 2004). We replicated the “conflict network” effect in all these populations, demonstrating the platform’s capability to run reaction-time-sensitive experiments - the effect we replicated was under 100ms.

We also have a pre-print that focuses on timing accuracies on various online platforms, browsers, and devices. You can read it here.

UCL are using Gorilla for teaching first year undergraduate students. One of the first research methods lectures was on the IAT. 108 participants took part in an IAT that is included in the Starter Project: Examples project folder. This IAT investigates whether participants have an implicit association between young people and good words, and old people and bad words.

The results of this test replicated previous findings that the participants do have such an implicit association.


Below are some highly recommended resources to help you learn more about timing in online research:

  • This article, Woods et al., 2016, is an excellent summary of the strength and weakness of online research, and contains a detailed section on timing.
  • This article, Hilbig, 2015, presents the effect of lab- versus web-based research on reaction times.
  • Our paper (2019) discusses timing accuracy in Gorilla.
  • Our recent pre-print (2020) compares timing accuracies on various online platforms, browsers, and devices. This talk presents the results from the pre-print and discusses the accuracy and consistency of visual presentation and recording of keyboard responses online.
  • Learn more about your participants and devices they are using from this article, written by one of our Gorilla Developers Alex Anwyl-Irvine on the Gorilla blog.