Support Home

How To: Participants

  • Overview
  • Participant Status and Progress
  • Changing Participant Status
  • Participant Tokens
  • Rejecting Participants
  • Including Participants

Overview


Welcome to the Participants 'How To' Guide!

Navigate through menu to the left to learn about different Participant statuses in your Experiment, how these affect your tokens and how you could manually change them when needed.

See the Participant Tokens section below to learn when participants tokens are consumed and visit our Pricing FAQ for further information on token prices, lab and departamental subsciprtions.


If you prefer to watch a video, see our Video Walkthrough that covers the contents of this page and shows a live example on how to Reject and Include Participants in your Experiments.

Participant Status and Progress


Participant Status

In your Experiment Tree > Participants Tab the Status column refers to how far your participant has moved within the experiment.

Live - the participant is still taking part, or has dropped out (i.e. did not reach the 'Finish' node).

Activated - the participant was activated using specific recruitment polices (Email ID, Email Shots, Supervised recruitment).

Complete - the participant has completed the experiment (i.e. reach the 'Finish' node; their data is stored and ready to download).

Rejected - the participant has reached the Reject Node, or was manually rejected.

RejectedTime Limit - the participant has reached the Time Limit.


Participant Progress

You can further monitor how far the participant has gotten through your experiment by clicking ‘View Progress’ under the Progress column.

Based on the participant progress you can make a decision on whether to Include or Reject Live participants.

To make educated decisions about whether include or reject a 'Live' participant, we strongly recommend you make use of Checkpoint Nodes in order to know how far through your experiment your participant progressed. See sections below for more details on how to Include and Reject participants.


Changing Participant Status


There are certain cases in which you can manipulate Participant Status. This can be done by choosing one of the options from the Actions column in the Participants tab.


'Complete' Participants

Participant Status cannot be changed for participants who have Completed your experiment because they have consumed your Tokens, i.e. participants with the Complete status cannot be further Included or Rejected - see the image below.

Note: Participants with the Complete status could be Deleted, however, we strongly advise you not to do that as you will permanently lose your data. The Delete option should only be used to comply with your ethics, i.e. to withdraw data from participants who no longer wish to be a part of your experiment or to remove participants data after the time you specified in your ethics application. The Delete option could also be used for deleting pilot data that you used for testing your experiment but we warn you to use it carefuly not to lose your valid data. Deleting participants does not reclaim your Tokens as these have already been consumed.


'Live' Participants

You cannot manually change a participant from Live to Complete - participants are only marked as Complete if they reach the Finish Node. However, you can manipulate Participant Status for Live participants by Including or Rejecting them in order to either use their existing data in your analysis or release the tokens they are temporarily occupying.

Learn how to manipulate Live participants from the Rejecting Participants and Including Participants sections below.


Participant Tokens


When a participant enters the experiment (reach the Start Node), they reserve a token.

The token represents the data that can be downloaded.

Participant Tokens can be either ‘Fulfilled’ or ‘Reserved’. Participants who have been Included (Complete, Manually Included, or Included at start) are considered ‘Fulfilled’, while all Live or Non-Included participants are considered ‘Reserved’.

When you have a Recruitment Target set, the experiment will continue to accept participants until the combined number of fulfilled and reserved participants reaches your recruitment target - the Recruitment Status is then 'Full'. If any participant holding a ‘Reserved’ token is subsequently Rejected (either by a Reject node, a Time Limit or manual exclusion), their reserved token will be returned, and the experiment will subsequently accept another participant until the total is reached. However, their token will not be returned if they have previously been Included. When the total number of Recruitment Target is reached, the Recruitment Status will then be 'Complete'.


When are tokens consumed?

Gorilla charges one participant token per participant per experiment.

When a participant starts your experiment, the Start Node reserves a token for them. When that same participant enters the Finish Node that token is then consumed, i.e. the participant is paid for. The token this participant consumed is then permanently removed from your accounts token total.

If the participant does not enter the Finish Node, and does not pass through a Reject Node, the participant's token will remain reserved until they are Included or Rejected (either manually or via a timelimit).

Tokens are consumed and therefore will be paid for when:

  1. Live participants complete the experiment, i.e. reach the Finish Node after which they will be marked as Complete.
  2. You manually Include Live participants after which their available data can be downloaded for analysis.
  3. You manually Include previously Rejected participant.

Data from participants who consumed your token is available for you to download it for the analysis.

Tokens are not consumed when:

  1. Live participants are Rejected, unless you manually Include them into your data after they have previously been rejected. After a Live participants is rejected, a token that they were temporarily occupying is released and you can recruit another participant in their place.

Further information

See our Pricing FAQ for more information on tokens, prices, lab and departamental subscriptions.


Rejecting Participants


Sometimes participants will not meet your experimental requirements and for these participants you may not want to consume a token. For these cases Gorilla allows you to 'reject' participants.

Rejected participants are:

  • Marked as 'Rejected' on your participants Page.
  • Not counted towards your Recruitment Progress.
  • Not included in your downloadable data.

Rejected participants can still be manually included in your downloadable data if you wish, find out how here.

Please note that consumed tokens cannot be reclaimed by later rejecting them, i.e. if your participants completed the experiment or you manually Included them in your dataset their tokens have been consumed and cannot be further reclaimed. Find out more on when tokens are consumed from the Participant Tokens section below.


How do I Reject Rarticipants?

To make educated decisions about whether or not to reject a 'Live' participant, we strongly recommend you make use of Checkpoint Nodes in order to know how far through your experiment your participant progressed.

If after looking at the Participant Progress and Status you have decided that a Live participant does not fulfill your study requirements, has dropped out or is taking longer to complete your experiment than is considered reasonable (see the IMPORTANT Note below), you can reject them. After participant is rejected, the token they were occupying is released and you can recruit another participant in their place. Read more about Tokens in the Participant Tokens section below.

Participants can be rejected (and therefore stopped from consuming your token) in the following ways:

  1. When they reach the Reject Node in your Experiment Tree design.
  2. When they reach the Time Limit set up on your Experiment Recruitment page.
  3. Manually by you - the experimenter, on your Experiment Participants page.

See more information about each of the rejection methods below.

Rejected participants can be later Included in your data - see the information at the bottom of this page.

IMPORTANT Note: To ensure you won't lose your participants, make sure that you allow enough time for participants to complete the experiment. You can do this easily by setting a Time Limit for your experiment. The time allowed will vary between the experiments - it could vary from hours to days to even longer time if you are conducting a logitudinal experiment. The decision is yours and will depend on the nature of your experiment but we generally recommend that you allow participants at least 3 times the expected completion time to complete your experiment, e.g. if you calculated that your experiment should take an hour to complete, allow at least 3 hours before you reject participants who exceed this time.


Reject Node

If your experiment has specific participant-requirements, you may wish to screen participants out early on in your experiment (for example, based on demographic questions or perfomance in a task) and not include them in your downloadable data. This can be achieved by directing participants into a Reject Node (instead of a Finish Node) in your Experiment Tree design.

Any participant who ends your experiment at a Reject Node will be given the status Rejected on your Participant Page. Their reserved token will be returned and they will not count towards your recruitment progress. They will not be included in your downloadable data. Previously reserved tokens will be reclaimed back into your experiment, so they can then be reserved and consumed by another participant.

Learn more on how to set the Reject Node for your experiment here.


Time Limit

The classic use-case for Time Limit Rejection is to reduce number of participants with 'Live' Status who temporarily occupy your tokens. To do that, you may wish to set a Time Limit on your Experiment. This means that participants who take more than the allowed time will be automatically rejected. This will prevent you having to manually reject participants who are still marked as 'Live' because they left the experiment part-way through.

Participants who reach the Time Limit will be automatically rejected and will be given a status of RejectedTime Limit on your Participants Page, but will be allowed to finish their current task, before being redirected to the Finish Node. Their reserved token will be returned and they will not count towards your recruitment progress. They will not be included in your downloadable data. Previously reserved tokens will be reclaimed back into your experiment, so they can then be reserved and consumed by another participant.

Learn more on how to set the Time Limit for your experiment here.


Rejecting Participants Manually

Sometimes participants will leave your experiment part-way through. On your experiments 'Participants' page, the status of these participants will remain as 'Live'.

While a participant has 'Live' status you can choose to manually 'reject' the participant from your experiment if you wish. Participants can be rejected manually by selecting 'Reject' from the Action column in the Participants Tab of your Experiment as shown on the image below. Rejected participants will be given a status of RejectedManual on your Participants Page. Their reserved token will be returned and they will not count towards your recruitment progress. They will not be included in your downloadable data. Previously reserved tokens will be reclaimed back into your experiment, so they can then be reserved and consumed by another participant. Once a token has been consumed however, it cannot be reclaimed in this way.


Including Rejected Participants

After the participant is rejected by one of the ways described above, you can still later include them and use their data for your analysis. Including participants will consume a token, i.e. you will pay for them. Find out more about Including Participants in the next section.


Including Participants


There are two classic cases where you may wish to Manually Include a participant:

  1. Manually including 'Live' Participants
  2. Manually including previously rejected participants

Choosing to Manually Include a Participant will:

  • Mark the participant as Included on your 'Participants' page.
  • Include this participant in your downloadable data.
  • Count the participant towards your Recruitment Progress as 'Fulfilled'.
  • Immediately consume a token from your experiments allocation.

Including a participant is a permanent action; once a participant has been included in your downloadable data, and a token consumed, the decision cannot be reversed. Be sure to only include participants whose data you wish to download and use.

To make educated decisions about whether or not to manually include a participant, we strongly recommend you make use of Checkpoint Nodes in order to know how far through your experiment your participant progressed.

Note: If your recruitment progress is at 100%, you must first increase your Recruitment Target before you can continue to manually Include participants. For Pay-per-Participant accounts this allows you to keep tight control of how many tokens you spend on an experiment.


Including 'Live' Participants

You may choose to include participants who have the 'Live' status at any time. This will allow you to download their data and view it while they are still progressing through your experiment. This can be a useful in Teaching situations or Supervised trials using the 'Supervised' Recruitment Policy.

You can include Live participants by selecting 'Include' option from the Action column as shown on the image below. This will immediately consume one token per participant from your experiments allocation.


Including Rejected Participants

Sometimes you may wish to include 'rejected' participants in your data collection even though they did not complete your experiment, or else ended your experiment at a 'Reject Node'. Even though you have marked these participants as 'rejected' you always retain the option to Manually Include these participants in your downloadable data.

To include participants that you have previously Rejected select 'Include' option from the Action column as shown on the image below. This will immediately consume one token per participant from your experiments allocation.


After you've included the Live/Rejected participant, you will be able to see a green tick in the 'Included' column, as shown below. You will then be able to download data from these participants for your analysis. Included participants permanently consume your tokens.