How to: Metrics


Overview


Find out more about metrics here.

Accessing your data


All data you collect is owned by you. Data can be downloaded from the Data tab in the Experiment Builder Tool.

Respondent data is stored by version, meaning that if you make changes to your experiment during data collection, data from respondents who completed the earlier version of the experiment will be stored separately from data from later respondents.

To download your data, use the Version Picker to select the version of your experiment that your participants completed.



Once you have selected the correct version, you can choose to download your data by-node, or all together in a zip file (see the tab 'Data separation').

To download all your data, click the button labelled Manage Experiment Data.

To download data by node, click on that node.



You can then select what file type you’d like your data to be downloaded in (XLSX, CSV (comma, semicolon or tab delineated), or ODS).

You have the option to unblind your data, which displays Public IDs along with Private IDs in your metrics, meaning that your participants could potentially be identified. You can also choose to display your questionnaire data in short form (see the tab 'Data Format').

Select the time period of data you’d like to collect, then click Generate. Generating data does not affect your token number, and you can generate your data an unlimited number of times.




Once your data has been generated, click Download. If your data takes longer than a couple of hours to generate, please contact us.

If your data file is blank, check that participants have completed the version you’ve downloaded data for, and that these participants have been included.

Data Separation


It’s good practice to keep performance data, demographic data and identifying data separately, which is why we give you a separate file for each node. You can combine data from tasks into a single file, and data from questionnaires into a single file.

We have a walkthrough available on how to combine these data files here.

Data Format


You can download data in XSLX (Microsoft Excel Open XML Format Spreadsheet file), CSV (Comma Separated Value), CSV (Semicolon), CSV (Tab) or ODS (Open Document Spreadsheet) formats, which can be easily imported into your favourite data analysis software.

We provide task data in long format, one row per trial. This can easily be converted into short format one row per participant using Pivot Tables in Microsoft Excel.

View our 'Pivot Table' video walkthrough to find out how!

Questionnaire data can be downloaded in either long-format or short-format (one row per-participant).

Note: If you have used a Script widget in your questionnaire, you must download your data in long-format in order to see the metrics it generates.

Understanding Your Data


When you first open your data-sheet, it can be intimidating to try and find the metrics you're looking for. Below are some data-sheet examples, with the location of different types of data.

This is what long-format questionnaire data looks like (click to expand):

Columns A-AL are Experiment data including participant information and the participants nodes they passed through. Columns AM-AO are Questionnaire and Response data. Experiment data columns are included in all task and questionnaire spreadsheets.

This is what long-format task data looks like (click to expand):

Columns A-Al are Experiment data including participant information, and the nodes they passed through. Columns AM-BA are Task and Response data, and columns BB-BE are the Task Spreadsheet. Experiment data columns are included in all task and questionnaire spreadsheets.

Most Experiment data columns are included in all downloads for completeness, as many are there for informational purposes only. Some columns will only be present if their corresponding Node is present in the Experiment Tree. Similarly, in short-format Questionnaire data, the columns presented are dependant on the widgets used in the Questionnaire. The next page includes a list of the columns that may appear in your data sheet, and what the values within them represent.

Data Columns


This page lists the columns that may be present in your data download, and what the values within them represent. Experiment data columns are present in both Questionnaire and Task data files.


Experiment Data Columns

Column Name Description
Event Index This is a counter created by the
UTC Timestamp The time in UTC milliseconds (UTC x 1000) on the server when this metric was received
UTC Date The timestamp above, in a human readable format
Local Timestamp The time in UTC milliseconds (UTC x 1000) on the client when this metric was recorded
Local Timezone The time difference (in hours) between the client’s local time and UTC
Local Date The Local Timestamp above, in a readable form, expressed in the Local Timezone
Experiment ID Unique key identifying the experiment
Experiment Version The version of the experiment
Tree Node Key The unique key for this tree node. This allows you to combine files without losing track.
Repeat Key The unique key for the repeat node. This key takes the form of the two repeat node components that form the Repeat Node. e.g. repeat-keyxyz#2->repeat-keyabc#1. This allows you to combine files without losing track.
Schedule ID Unique key identifying the schedule, which corresponds to a particular participant performing the task or questionnaire associated with a particular tree node
Participant Public ID Unique ID representing this participant. This is visible on the participant’s page, and so is hidden for blinded experiments
Participant Private ID Unique ID representing this participant
Participant Starting Group The group that the participant started in. Used for experiments with multiple start nodes.
Participant Status The participant’s completion or rejection status.
Participant Completion Code Completion code that was shown to the participant. Used for validating completions on third party recruitment services (e.g. MTurk)
Participant External Session ID External session ID provided by third party recruitment services (e.g. Prolific)
Participant Device Type This column gives information about a participants device type, this will be either, 'computer', 'mobile' or 'tablet'.
Participant Device This column gives more detailed information about a participants device, the detail of infomation here will depend upon the device and settings of the device owner. For example, if available will list type of mobile device in use.
Participant OS This column gives information about a participants Operation System (OS) e.g. Windows 10
Participant Browser This column gives information about a participants browser type and version.
Participant Monitor Size This column gives information about a participants monitor size in pixels: width x height.
Participant Viewport Size This column gives information about a participants viewport size in pixels: width x height. The viewport size is the effective size of a browser window, minus the browser header bar and any OS navigation bar at the top and bottom or sides of the screen. The height should be smaller than the monitor size, but the width is usually the same size.
The value should also stay the same size throughout a participants experiment, if it does not it indicates that a participant is resizing the window during your experiment a possible indication of divided attention which you may wish to factor into your analysis.
Checkpoint Name of the last checkpoint that this participant went through
Task Name The name of the task
Task Version The version of the task
randomiser-a8ul The branch that this Randomiser node assigned the participant to
branch-213o The branch that this Branch node assigned the participant to
order-skbx The order that this Order node assigned the participant
switch-rkki-time-primary This is the total time (in ms) the participant spends on a primary switch task.
switch-rkki-percentage-primary This is the time the participant spent on a primary switch task displayed as a percentage.
switch-rkki-time-secondary This is the total time (in ms) the participant spends on the secondary switch task.
switch-rkki-percentage-secondary This is the time the participant spent on the secondary switch task displayed as a percentage
switch-rkki-switches This is a count of the total number of switches a participant made between the primary and secondary tasks.
counterbalance-kczp The spreadsheet column used by the counterbalance node.
checkpoint-jzwa Each Checkpoint Node produces its own column. When a participant passes through the Checkpoint, the name of the Checkpoint will appear in its column.
quota-keyidentity The status of the Quota Node that this Quota Node assigned to the participant.

Task Data Columns

Column Name Description
Trial Number The trial number for this trial. Trial numbers increment every time the task builder task moves on to a new row in the spreadsheet.
Screen Number Screen number within the current display
Screen Name Screen name within the current display
Zone Name Name of the zone that generated this metric
Zone Type Type of the zone that generated this metric
Reaction Time Time between the start of the current screen and when this metric was generated
Response The response that was given, if any
Attempt Which attempt at the correct answer this response represents (used when multiple responses are enabled)
Correct Whether this response was judged as correct
Incorrect Whether this response was judged as incorrect
Dishonest Produced by the Feedback (Accuracy) Zone. A 1 in this column indicated that dishonest feedback was given.
X Coordinate If using the Click Painting Zone, This will be the position of the click (X coordinate) relative to the original image (regardless of rescaling).
Y Coordinate If using the Click Painting Zone, This will be the position of the click (Y coordinate) relative to the original image (regardless of rescaling).
Timed Out Whether this response was given as a result of a time out (rather than action on the part of the participant)
Spreadsheet Name The name of the spreadsheet used
All remaining columns These are copies of the spreadsheet row shown to the participant

Questionnaire Data Columns (long format)

Column Name Description
Question Key The Question Key (i.e. Response-1). This Key may be -quantised – A numeric version of a text response i.e. the first option on a Likert Scale.
Response The response given by the participant

Questionannaire Data Columns (short format)

Column Name Description
Name-of-your-Question-Key Response to a widget. In the case of consent boxes, 1 indicates consent.
(Name-of-your-Question-Key) -text If your question has an ‘Other (please specify) option, this column represents any typed response
(Name-of-your-Question-Key) -quantised If you are using a Dropdown widget, a Likert scale, or Radio buttons, this a number representing the option they selected. e.g. Option 1 would be represented as 1.
(Name-of-your-Question-Key) - 1 If you are using a checklist widget, this represents the first option you give on the checklist. If there is a response in this column, the participant has selected this option. If you are using a ranking widget, this represents the first ranked option
(Name-of-your-Question-Key) -2 (ect) If you are using a checklist widget, this represents the second option you give on the checklist. If there is a response in this column, the participant has selected this option. If you are using a ranking widget, this represents the second ranked option
(Name-of-your-Question-Key) -year For Date Entry widgets, contains the year given as a response.
(Name-of-your-Question-Key) -month For Date Entry widgets, contains the month given as a response.
(Name-of-your-Question-Key) -day For Date Entry widgets, contains the day (numerical - of the month) given as a response.
(Name-of-your-Question-Key) -hour For Time Entry widgets, contains the hour given as a response.
(Name-of-your-Question-Key) -minute For Time Entry widgets, contains the minute given as a response.
(Name-of-your-Question-Key) -mixed If you are using a Mixed-entry widget, this column will hold any ‘selected’ rather than typed responses. If your participant has typed a response, this will appear in a different column, and the -mixed column will be empty.
End Questionnaire The number of milliseconds it took your participant to complete the Questionnaire.

Time on Task


For each participant for each node, you will get additional rows to marks when participant BEGIN and END the task. This row can be identified because there the trial number will be BEGIN TASK or END TASK.

These rows will come with timestamps. Additionally, in the END TASK row you will also get a Reaction Time in the Reaction Time column that is the time since the BEGIN TASK timestamp. This can be used to calculate the time taken to complete the task.

Data Analysis


Task data is provided in long-format (one row per trial) in a format which can be easily downloaded as CSV files and pivoted into short format (one row per participant).

Questionnaire data can be downloaded in either long-format or short-format (one row per-participant).

You can then upload your Gorilla metrics CSV file into your preferred statistical analysis package e.g. SPSS, R, Excel etc.

Tips and Tricks for Excel:

Learn how to quickly clean and bulk-edit your data in excel: Download

Not sure how to use Pivot tables?

You can learn how to pivot your data using excel via our short you-tube video walkthrough - here.

A further walkthrough is available here.

Data Security


Our secure data storage, located within the EU is end-to-end encrypted and fully complies with Data Protection Regulation [EU & UK] (GDPR), the British Psychological Society (BPS) and National Institute for Health Research (NIHR) Guidelines.

You can find out more about data security and related questions in our F.A.Q: here