Experiment Requirements


Overview


Welcome to the Experiment Requirment Options Tooling Reference Guide.

Here you can learn about the different Requirements options available in a Gorilla Experiment, and about Time Limits you can set on Experiment completion.

Requirements allow you to restrict access to your experiment based on certain characteristics/conditions. There are four categories of restriction: Device Types, Browser Type, Location and Connection Speed. Any participant who doesn't satisfy the requirements you stipulate will be prevented from accessing your experiment. They will be sent to a default page explaining why they are not elligible to take part.

Requirements are set on a per Experiment basis. They can be viewed and changed from an experiments 'Recruitment' page.
In your Experiment, select the 'Recruitment' tab. In the bottom right-hand quarter of the tab you will see the Experiments current Requirements. By default, these are set to 'No Restrictions' in each category. To change the Requirements, select 'Change Requirements.'

To learn more about each Requirements category, select from the options on the left-hand side.

To learn more about participant recruitment and the options available, check out the How Do I Recruit Participants support page.

Device Types


By default, participants can take part on all web enabled devices. You can optionally limit by device types by selecting the options that you want to accept. You can accept:

  1. Computers (Desktops and Laptops)
  2. Tablets
  3. Phones

If participants try to access your experiment from a device type that is not allowed, they will reach a special page that tells them that their current device is not allowed and that they can try again later from a different device.

We use an external library to detect whether a participant is using a phone. This library has a list of phones which we then exclude. It's not - unfortunately - an exhaustive list so some phones do still get through.

Browser Type


By default, all browser types are allowed.

There can be small differences in how different browsers display certain things. For example, how a dropdown menu looks or operates when clicked on. Or else how special characters are displayed. There can also be differences in the accuracy (number of significant figures) a reaction time can be recorded in from different browsers.

For some research, where these small difference have little or no significant bearing on the hypothesis under investigation, which brower type a participant uses will not be important. In these cases you will not wish to restrict participants based upon browser type.

However, for other research, where the element in question is fundamental to your hypothesis and you have detected a noticible difference between browsers, you’ll want to ensure that all participants use the same browser or selection of browsers when they undertake your experiment. A typical example of this case is, if you are using a code task which uses cutting-edge features that are only available in a specific browser, and therefore won't work in other browsers, you will want to limit your participants to using only this browser.

Note: If the participant clicks on a link from within a mobile app, the experiment may open within the app, rather than in a full browser such as Safari or Chrome. If this occurs, participants may not be able to complete tasks. We strongly recommend you ask participants to copy and paste the link for use on mobiles, rather than clicking it.

Location


By default, anyone in any location can take part in your experiment.

You may want to limit respondents by country. You can do this by providing a list of countries from which you will allow participants.

Some people obfuscate their IP Address, so that the country cannot be detected, you can decide whether to allow these participants or not.

If participants try to access your experiment from a location that is not allowed, they will reach a special page that tells them that their current location is not allowed and that they can try again later from a different location.

Connection Speed


Gorilla uses a lookahead system to ensure that everything a trial needs to display properly (such as videos and images) are downloaded before the trial starts. This way, a participant's connection speed does not affect the accuracy of their reaction times.

The only scenario in which a participant may experience pauses between trials while the loading catches up is if it takes longer to load a trial than it does to perform one. This means that either their connection speed is too slow, or that the task is trying to show lots of large images or videos very quickly. You can solve this by either using smaller images or lower quality video, or alternatively by setting the required connection speed to be higher.

For the majority of cases, most modern connections will suffice, and so by default there is no restriction. However, if your experiment has some high-bandwidth elements, you should consider setting a minimum connection speed.

Note that the connection speed detected by Gorilla will be different to the connection speed that a participant may detect for themselves using a service like www.speedtest.net. Services like speedtest.net determine the maximum theoretical speed of an internet connection in ideal conditions. This is useful for establishing whether your internet service provider is indeed providing you with the right connection speed, but this is always going to be greater than the connection speed experienced during normal use.

The test carried out using the Gorilla Connection Speed Test will generally be a more accurate representation of the participant’s connection speed under current conditions. Nevertheless, this requirement option is provided as a BETA tool, as we have plans to improve it further.

If participants try to access your experiment using an internet connection that is too slow, they will reach a special page explaining this and that they can try again later when their connection speed is better.

Time Limits


Time Limit, found on your experiment recruitment page, allows you to automatically reject participants who do not complete your experiment, or who take longer to complete it than is considered reasonable.

Once a Time Limit is set, in hours and minutes, participants who reach the Time Limit will be automatically rejected, but will be allowed to finish their current task, before being redirected to the Finish Node. You may wish to set a Time Limit because ‘Live’ participants reserve tokens, contributing towards your recruitment target. This means that participants who drop out without finishing your experiment can prevent more participants from entering your experiment until they are rejected.

Whilst you can reject participants manually, this requires monitoring your recruitment progress closely. Instead, you may choose to set a Time Limit to automate this process.

We suggest setting a Time Limit that is far longer than it could reasonably take to complete your experiment. For example, If your experiment should take 15 minutes to complete, you might set your time limit at 2 hours.

For this reason, we do not recommend using Time Limits for longitudinal studies. In a longitudinal study, the reasons for taking a long time to complete a study are much more numerous, which makes the padding you'd want to give the time limit excessively large and hard to estimate. When you can see your attrition and rejection numbers, you may wish to revise your Time Limit, and would then have to manually include the participants you’d automatically rejected.

Additionally, depending on ethics and your recruitment service, you will likely still have to pay participants who only complete the first half of your study for completing the first section, so you may wish to make use of their data.

Note: When using the Time Limit with recruitment services that offer a similar Time Limit, make sure that your Gorilla experiment Time Limit matches the time limit set in the recruitment service.