Today many experimental psychologists and other behavioral scientists turn to task and experiment builders to help them create experimental protocols. But many don’t. They still prefer to spend the time writing their experiments in code.
So if you’re still writing code and you’re wondering whether to consider using a task builder, here are 10 reasons why using a task builder might be a good idea.
New tools have always preceded rapid innovation, because they make it easy for large numbers of people to do something that was previously restricted to a niche expert group.
Give ordinary people the right tools, and they will design and build the most extraordinary things. Neil Gershenfeld
Over the last few years, several tasks builders have emerged that can be used by experimental psychologists and economists to build behavioural experiments. For a review of tools available see here.
Many researchers have taken this leap. Why have they done this? And what are the benefits?
Using a task builder has a far lower barrier to entry. While coding is fab, and my inner puzzle solving, lego building child self just LOVES tinkering with code and making it COME ALIVE, I do recognize that for some people it’s a complete turn off.
Syntax, variables, if else statements just aren’t everybody’s jam. The same way that – for me – writing is a painful process – for others coding just isn’t their thing.
Task builder set you free from the painful side of coding. When they are done right, you can do things like:
… And you’re off… All the pain is gone.
And all you’re left with is the ‘it’s alive’ feeling.
Several universities are now using task builders as part of their undergraduate research methods course, so the barrier to entry is low. Not quite as low as eating chocolate, but… you get the idea!
Building a task in a task builder is undeniably faster. A simple 4 way forced choice experiment with image stimuli, might only take 15 minutes to create from scratch and deploy online. More time will be spent creating the stimuli than creating the technical implementation of the experiment. Which leads me onto my next point…
The economist in me likes to price the time saved. Let’s say you’re a post-doc earning £36,000 a year. In code, building your task might have taken you 1 month (£9k) or 2 weeks (£4.5k).
Using a task builder, even if this took you a whole day (£450) – you’re still saving £4,050 in opportunity cost.
You can now spend that time designing a better study, doing a better analysis, writing a better paper, or spending some time relaxing with friends or family, perhaps evening going to the gym or cooking yourself a delicious meal!
Have you ever inherited someone else’s code? There’s nothing worse, except maybe inheriting someone else’s notes. It’s often incomprehensible. To be honest, unless it’s very well commented, it might even be incomprehensible to the original author within a sufficient delay.
Task builders are different. Because you are working within a framework, you can clearly see the design choices that have been made. You may not understand the design choices, and you may disagree with them. But the choices made are clear.
Not only are they clear to someone with tremendous technical skill. They are also clear to someone with very little technical skill. So, task builders are both transparent and democratic.
Counter-intuitively, task builders provide more technical longevity. When you’ve written something in code, it will be interpreted by the target program (i.e. a browser) exactly as you have specified.
Perhaps you were writing your code back in the 1990s and you specified the layout based on the position (pixels from the top and pixels from the left). When screen resolutions increased, your code needs to be updated. In contrast, there is always a level of interpretation between a task builder and what the participant sees. As web standards change (which they will) and hardware changes (at a relentless pace) the provider of your preferred task builder will update how the specification is interpreted to ensure that your task continues to run.
One crucial question for behavioural scientists is which tool will give them the best timing. A good task builder will provide you with the best in class and validated timing (see timing paper). This is because those creating task builders (1) know what they are doing and (2) have the resources to validate their implementation.
In contrast, while it’s possible to get excellent timing in code, the responsibility falls to the individual and so is constrained by your technical ability. You may also not have the resources to validate your implementation as that requires specialist hardware and is time-consuming.
Those that create task builders are highly motivated to create a wide range of samples, so that their clients don’t have to reinvent the wheel. So it might be that your task is already built and ready to go!
Gorilla also provides an open access repository where researchers share their tasks for others to use. This provides a hugely efficient way of encouraging replications and building on each other’s work.
Similarly, those that create paid for task builders are highly motivated to provide excellent customer support.
Second, and more importantly, the support team at Gorilla is the best I have worked with in my research career. They respond promptly and thoughtfully to resolve any issues along the way in experiment building.. MVM, Notre Dame
This is a huge benefit. You have the equivalent of a specialist research assistant only an email, phone or video call away. When you get stuck and don’t know what to do, you don’t have to ask someone a favour and wait for them to be free. Instead, there is a team whose job it is to support you.
A good task builder will also often offer a consulting service. We are frequently hired by clients to add new functionality to Gorilla (eye tracking, canvas painting, AAT zone, real effort zone). This is also an excellent use of grant money as the feature not only benefits the original research team, but also becomes available to all other researchers using Gorilla around the world, which is currently over 10,000! The return on investment for the research community is tremendous.
So with all these benefits, why are some people still writing in code?
Are there any downsides to using task builders?
To answer this question, it’s useful to step back and think about the development process of a task builder. Those creating task builders will be adding UI (user interface) to provide functionality that is wanted by many people.
The efficiency comes because the functionality is written once by an expert research software developer, and used many times by researchers. However, behavioural science experiments sometimes need functionality that has never been used before. So, one of the trade-offs is getting exactly what you want through their GUI.
Nevertheless, the best task builders allow you to augment the functionality that they provide with your own bespoke code. This gives you the best of both worlds, a UI for aspects of your experiment that aren’t novel, and code for the novel elements.
There are two generic approaches to augmentation. The first is that you build your task in the GUI, then export it to code, and then make changes. This approach means that you can’t then go back to the GUI to make changes which can be frustrating.
The second, is to build your task in the GUI and then be able to add snippets of code that add additional functionality while staying in the task builder. This latter option is better as you are still able to make changes using the GUI in parallel with tweaking your bespoke additions. The latter is how Gorilla works.
But… Learning to code is valuable
Yes it is!
Imagine that you’re an undergraduate or master’s student in psychology, or behavioral economics. Given that you don’t need to, is learning to code in JavaScript the best use of your time? Or would it be better spent mastering the science and statistics components of your course? Or perhaps you have other uses for your precious time? Like earning money? Or seeing friends? Or pursuing hobbies? Exercising? Meditating?
If you really want to code, your time may be better spent mastering data analysis in R. Once you are confident with the basics of coding move onto augmenting functionality in a task builder. This approach makes sure that you are always getting a good benefit for the time you invest in learning to code
Building and maintaining a platform is complex
Building and maintaining a whole platform for running online tasks is complex technically and legally. You’d need to:
This might be too much for some to add to their plate.
Avoid this nightmare scenario!
And you definitely want to avoid the horror scenario that happened to one of our (now) clients. They spent 3 months creating their own server architecture out of open-source components to host their task. Their ethics board said they couldn’t use it, so they came to us to host it instead.
Ok – So I should use a Task Builder, but which one?
As far as we know, Gorilla is the only experiment builder on the market. Not only do we provide a task builder, we also provide an GUI experiment design tool, which makes it easy to design simple and complex experiment protocols. Our tool provides randomization, branching, counterbalancing, as well as repeat and delay nodes for multi-day studies.
Author
Jo is the CEO and co-founder of Cauldron and Gorilla. Her mission is to provide behavioural scientists with the tools needed to improve the scale and impact of the evidence-based interventions that benefit society.
Date Published: 9th Feb 2021
We regularly run grants to help researchers and lecturers get their projects off the ground. Sign up to get notified when new grants become available