Complex experiment designs made easy

Gorilla Experiment Design Tool

Gorilla is an experiment design tool, not just a task build­ing or ques­tion­naire tool. This is one of the biggest dif­fer­ences between it and other tools avail­able for hosting online exper­i­ments. The Experiment Build­ing tools allow you to cre­ative­ly recon­fig­ure task and ques­tion­naires into a wide number of dif­fer­ent experiment designs without having to code.

The inter­face is built around drag­ging and drop­ping nodes and con­nect­ing them togeth­er with arrow lines. This modular approach makes it much easier for labs to reuse ele­ments that have been created before by them­selves or by others.


Simple Experiment

This is the sim­plest struc­ture in the Experiment Builder, the start and finish ‘nodes’, indi­cate where the par­tic­i­pant enters the experiment, and when they end. The green nodes are two sequen­tial ques­tion­naires (for gaining consent and demo­graph­ics), and the blue node rep­re­sents a task built in the Task Editor or Code Editor. The (vX) in the brack­ets of the task node indi­cates the version of the task, as the builder imple­ments a version control system allow­ing you to roll-back your task to any pre­vi­ous saved version.


Within-subject design

Expand­ing on the simple experiment struc­ture above, this rep­re­sents a typical within-subject design. After consent and demo­graph­ic ques­tion­naires, the tasks con­tain­ing the two depen­dent vari­ables are split up into two task nodes – ‘Rela­tion­al Rea­son­ing’ and ‘Affect Test’.


Within-subject design with order control

This is a slight­ly more complex version of a within-subject design, it makes use of the order node, which allows the exper­i­menter to control for the order of the tasks between par­tic­i­pants. The order node allows for a stan­dard ‘Latin Square’ design – where all orders in a square are shown equally , or a ‘Bal­anced’ design – where all pos­si­ble per­mu­ta­tions are shown (note: there is only a dif­fer­ence between these choices with four or more tasks).


Between subject design with randomizer

Between subject designs are also made pos­si­ble in the Experiment Builder, uti­liz­ing the ‘Ran­domiser’ node. This node will assign each par­tic­i­pant to one of n number of branch­es (in this case two, named control and treat­ment), whilst enabling the user to set the likelihood/weighting of each branch via a ratio field (in this case there is a 10:10 ratio, so for every 20 par­tic­i­pants 10 will end up in each branch). The node also has a choice of ran­dom­iza­tion mode, which can be ‘Bal­anced’ so that each node will get a fixed pro­por­tion of par­tic­i­pants (i.e. random without replace­ment), or ‘Random’ where the ratio acts a prob­a­bil­i­ty and equal pro­por­tions are not guar­an­teed (i.e. random with replacement).


Inter­ven­tion design with pre- and post- test

This example illus­trates an inter­ven­tion design with a pre- and post- test. The design is similar to the between subject design above, which pseudo-ran­dom­izes allo­ca­tion to control and treat­ment con­di­tions between par­tic­i­pants. The same test is given before and after com­plet­ing the assigned task, to ensure this test is not the same, two stim­u­lus sets are assigned (‘Stim­u­lus Set A’ & ‘Stim­u­lus Set B’), and these are bal­anced between par­tic­i­pants using the order node.


Inter­ven­tion design with screen­ing question

This example is the same as above, but uti­lizes a ‘branch’ node, which allows screen­ing out par­tic­i­pants before they com­plete an aspect of your experiment. A participant’s response to a ques­tion about smoking in the demo­graph­ic ques­tion­naire is pre­served using a func­tion called ‘embed­ded data’ and is used in the ‘branch’ node to ensure only smokers com­plete the tasks. Non-smokers are taken to a finish node, meaning they end the study there. Embed­ded data can be any­thing, from a survey response to an accu­ra­cy rate in a task – per­mit­ting complex response depen­dent experiment design.


Multi-day train­ing pro­to­col design

The ‘Repeat’ and ‘Delay’ nodes allow the user to create a multi-day train­ing pro­to­col. This example demon­strates how this would be done in a minimal way. After com­plet­ing the consent and demo­graph­ics ques­tion­naires, par­tic­i­pants enter a repeat loop (the dotted line between the two ‘Repeat’ nodes). At the end of each iter­a­tion, a ‘Delay’ node is con­fig­ured to send a message to the par­tic­i­pant asking them to come back in 24 hours, and it is also con­fig­ured to send a custom email at that time to the par­tic­i­pant with a link to con­tin­ue the experiment. This iter­at­ed three times, so each par­tic­i­pant is able to com­plete the same task three times before fin­ish­ing the experiment. Repeat nodes could be put around the inter­ven­tion design above, to create a multi data Ran­domised Con­trolled Trials, all without writing code.

Achieve your ambitions

So, what’s the take away? Gorilla is a pow­er­ful experiment design tool. Rather than weeks if not months of coding an experiment, you can create one in hours or days. By saving you time, we aim to acceller­ate your research cycle and help you achieve your ambitions.