Support Home

FAQ

  • Why Gorilla
  • Online Timing Accuracy
  • Anonymity and Ethics
  • GDPR: General Data Protection Regulation
  • Data Protection and Security
  • Server Downtime
  • The Code Editor
  • Licensing
  • Teaching
  • Gorilla Around the World
  • Ethics Applications
  • My Account Settings
  • Citing Gorilla

Why Gorilla


We were chatting to a client, and in a moment of frustration with their current tools they said 'I just want something as easy to use as Survey Monkey, for putting reaction time tasks online. Is that too much to ask?'

To which we responded 'Oh, so you want a bigger and better monkey! Maybe a Gorilla?'

The name stuck.

(Pedants: yes, yes, Gorillas are apes, not monkeys. Five points to Griffindor!)

Online Timing Accuracy


One of the most frequent questions we get is about the timing accuracy of Gorilla. The short answer is that we use high resolution timers for accurate reaction times and frame counting for accurate stimuli presentation times - techniques that have only recently become available in all major browsers.

In 2019, we published our paper on timing accuracy in Gorilla. You can read it here.

Recently, we have also uploaded a pre-print where we examined the timing accuracies on various online platforms, browsers, and devices. You can read it here.

See our Timing in Gorilla website for a detailed technical overview of the timing techniques employed in Gorilla and to learn more about timing in online experiments through some extra resources.

Anonymity and Ethics


In compliance with BPS (The British Psychological Society) requirements, identifying data, demographic information and performance data are all stored separately. They are downloaded separately from the metrics tab and joined together outside Gorilla using the Private IDs provided.

Our database architecture supports double-blind studies; you can join demographic data with performance data while remaining blinded.

If using Gorilla in conjuction with a third party recruitment service, it may be that you do not collect any identifying data.

GDPR: General Data Protection Regulation


Gorilla is fully compliant with GDPR.

Gorilla is built around the existing BPS (The British Psychological Society) and NIHR (National Institute for Health Research) standards which were far more stringent than the Data Protection Act. Moreover, GDPR does not apply to data that “does not relate to an identified or identifiable natural person or to data rendered anonymous in such a way that the data subject is no longer identifiable.” The majority of our recruitment policies anonymise participants.

Data Protection and Security


Gorilla is fully compliant with data protection and security policies.

Hosting: Gorilla is hosted on Microsoft Azure. * Currently, all our instances are located in their North Europe region, which is within the EU (Republic of Ireland). * In future, we may need to expand to other regions as we take on more international clients. This will allow us to keep data storage to particular jurisdictions if that's required from a legal standpoint * Microsoft Azure is compliant with ISO/IEC 27001:2005. More details.

  • Traffic Encryption: All traffic to and from Gorilla is encrypted (TLS/SSL)
  • Database Encryption: The database is encrypted using industry-standard cryptography
  • Ownership: The experiment owner owns the research data that has been collected using Gorilla
  • Data Protection: Participant research data can be fully deleted by the researcher.

Responsiblity for deletion of participant data falls to the researcher including accidental deletion of participant data.

Once data has been deleted it cannot be recovered.

Researchers are able to delete all data for an experiment or data pertaining to an individual participant. When this action is taken, data will be removed immediately from the database, and cleared permanently from our automated backups after 14 days.

GDPR: Gorilla is fully compliant with GDPR.

Server Downtime


Microsoft Azure guarantees that our servers will be working 99.95% of the time. There are 525,600 minutes in a year. That 0.05% when our servers could be down - outside of our control - equates to ~263 minutes a year. This is equivalent to ~2 minutes a month or ~44 seconds a day. At scale, very rare events happen surprisingly often.

Microsoft Azure performs far above this threshold, nevertheless server downtime is a reality of internet research, and we want to give you the information you need to make an informed decision.

  • Short server downtime: Some server downtime will be so short that it will not affect your participants. The participants computer will have stored a few trials ahead, so it’s possible that the server is back up again before the participant needs more information.
  • Longer server downtime: Some server downtime will be long enough that participants notice. It may still be sufficiently short that the participant has to refresh their browser, and in that case it might simply be a question of excluding a trial with a long reaction time. In this situation, the server downtime has impact similar to your participant being momentarily distracted.
  • Critical server downtime: Some server downtime will be long enough that participants cannot continue at that time. Depending on the recruitment policy and experiment, it might be that they can continue later. On the other hand, it might be that for experimental reason, you can’t use the data.

Due to the reality of the possibility of server downtime, we recommend launching experiments in small enough batches that you can afford to lose every participant that is currently active. On our side – as long as you haven’t included participants at the start node – no Gorilla fees would be due. If you are paying participants through a participant recruitment service, you may need to check their policy.

The Code Editor


Code in Gorilla is written in TypeScript, which is a typed superset of JavaScript that compiles to plain JavaScript. The reason we chose TypeScript over plain JavaScript is that TypeScript offers a wealth of useful features that make writing code easier, and because many of those features are destined to become part of the ES6 standard which is currently being ratified. This means that JavaScript will effectively become TypeScript, and so your code is future-proofed.

In terms of libraries, JQuery and Bootstrap are included by default.

Go here for more information about the code editor.

Licensing


We have a seat licensing model. Each person signs up with their own email address and effectively has their own account. Each user then has complete control over any task, questionnaire, experiment and associated data that they have authored. This model fits with BPS (The British Psychological Society) requirements around data security; data is only accessible by the person that owns the experiment or those that they are collaborating with.

Users can also collaborate on projects. When sharing projects the level of access (read, write, admin) can also be set.

We don't currently have the idea of student accounts and supervisors. Any account holder is able to publish their experiments and the onus is on them to ensure they have done so in compliance with their institutions ethics and code of conduct.

What happens when my licence expires?

When your licence expires, your account will revert to a Pay-per-Participant account. All you data, task, questionnaires and experiments will be maintained. You will still have access to all the editing tools and the previewing tools. You just won't be able to collect more data without first purchasing pay-per-participant tokens.

Teaching


Gorilla is an ideal environment for teaching Research Methods, as students can get valuable experience in operationalising experiments, collecting data, and analysing the data collected.

The Experiment Tree makes the experimental design clear, which can often help students understand whether their experiment is adequately controlled.

  • Getting Started: When students are getting started it can be useful to provide them with an experiment where they can change the preconfigured manipulations. For instance, they may change the timing of a task.
  • Beginner: Next, students may start to tweak tasks an questionnaires. They may use different stimuli, to answer a different question. Or add questions to look at alternative correlations.
  • Branching Out: Next, students may start to author their own novel tasks in the code builder.
  • Expert: Finally, students may start to use the Scripting Tools or Code Editor.

We have a suite of tools that allow teachers to manage classrooms. These allow you to:

  • upload lists of class members
  • share resources with them, and
  • receive submissions from them

If you want your students to try out a task for free (i.e. without using your tokens), you could add your students as collaborators to your project and they will be able to try it out for free using the preview. If you want to collect their data, they will need to download it when the option comes up at the end of the task/experiment, as it is not saved.

For Masters students who may not have the time or inclination to learn to code, Gorilla offers a user-friendly environment in which to author completely novel tasks.

Our blog entry from December 2016 describes the UCL experience of teaching 1st year undergraduates with Gorilla.

Gorilla Around the World


Gorilla supports a full range of characters sets in Tasks and Questionnaires including, but not limited to, Chinese, Arabic and Hebrew, so you can interact with participants around the world.

If you want to include text in a language that reads right-to-left, in a rich text or HTML zone, you can to move your punctuation to the beginning of your sentences by using the unicode character ‫.

You can also add <div style="text-align:right;">your text here</div> to align your text to the right.

Task spreadsheet cells will also accept formatting.

Ethics Applications


We're often asked to provide draft text for an ethics application.

Gorilla

We will use Gorilla (www.gorilla.sc) to collect data for our study. Gorilla is a cloud software platform specifically for the behavioural sciences.

  • Hosting: Gorilla is hosted on Microsoft Azure within the EU (Republic of Ireland)
  • Traffic Encryption: All traffic to and from Gorilla is encrypted (TLS/SSL)
  • Database Encryption: The database is encrypted using industry-standard cryptography
  • Data Ownership: The experiment owner owns the research data that has been collected using Gorilla and has complete control over it
  • Data Protection: Gorilla is fully compliant with data protection legislation
  • BPS: Gorilla is fully compliant with BPS guidelines.
  • GDPR: Gorilla is fully compliant with GDPR.

Recruitment

  • Data Collection: Participants will take part via a desktop computer, laptop, tablet or phone from [anywhere in the world]
  • Consent: Participants will give consent within Gorilla [see supporting documentation]. Participant can opt to not give consent.
  • Recruitment Policy:
    • We will use an anonymous recruitment policy in Gorilla. Consequently, once data is collected it cannot be deleted as it cannot be identified. Participants can still withdraw from the experiment at any time by closing their browser.
      or
    • We will use a recruitment policy in Gorilla that provides participants with a unique an non-identifyable key [ABC123456] that allows them to withdraw their data after completing the experiment.
      or
    • We need to collect data from participants over several days and therefore want Gorilla to email participants to remind them to take part. Consequently participant email addresses will be uploaded to Gorilla.
      • To ensure complete confidentiality and data security, participants are first given a Public ID (ABC123456) which they can use to log in with.
      • Performance data is stored against a Private ID (X1Y2Z345).
      • The relationship between the email address and Public ID is stored separately from performance data.
      • The relationship between the Public ID and Private ID is stored separately from performance data.

My Account Settings


Below you can find the answers to some most frequently asked questions about the user's account.

  • Can I change my email address or password?

    Yes! Go to your My Account page (in the right top corner menu) and then click on the Authentication tab.

  • Where can I find sign up offers I have received?

    Go to My Account page, Billing tab.

  • How do I change my email preferences?

    Go to My Account page, Mailing Lists tab.

Citing Gorilla


To refer to Gorilla in an ethics application, grant application or article for publication, please link to the main website. We also recommend stating the date window within which data was collected, so that someone reading the study could cross-references this with our release notes.

Example Text

We used the Gorilla Experiment Builder (www.gorilla.sc) to create and host our experiment (Anwyl-Irvine, Massonnié, Flitton, Kirkham & Evershed, 2018). Data was collected between 01 Jan 2017 and 15 Jan 2017. Participants were recruited through [Facebook / Prolific / Research Now].

Citation:

Published Paper (Preferred Citation)

Anwyl-Irvine, A.L., Massonié J., Flitton, A., Kirkham, N.Z., Evershed, J.K. (2019).
Gorilla in our midst: an online behavioural experiment builder.
Behavior Research Methods.
Doi: https://doi.org/10.3758/s13428-019-01237-x

Pre-Print

Anwyl-Irvine, A., Massonnié, J., Flitton, A., Kirkham, N. and Evershed, J. (2018).
Gorilla in our Midst: An online behavioral experiment builder
bioRxiv, 438242
doi: https://doi.org/10.1101/438242

Publications and Preprints

A list of publications and pre-prints that cite Gorilla can be found here.