Today many exper­i­men­tal psy­chol­o­gists and other behavioral sci­en­tists turn to task and experiment builders to help them create exper­i­men­tal pro­to­cols. But many don’t. They still prefer to spend the time writing their exper­i­ments in code.

So if you’re still writing code and you’re won­der­ing whether to con­sid­er using a task builder, here are 10 reasons why using a task builder might be a good idea.

Why use a task build­ing tool rather than writing code from scratch?

New tools have always pre­ced­ed rapid inno­va­tion, because they make it easy for large numbers of people to do some­thing that was pre­vi­ous­ly restrict­ed to a niche expert group.

Give ordi­nary people the right tools, and they will design and build the most extra­or­di­nary things.

 

Neil Ger­shen­feld

Over the last few years, several tasks builders have emerged that can be used by exper­i­men­tal psy­chol­o­gists and econ­o­mists to build behav­iour­al exper­i­ments. For a review of tools avail­able see here.

Many researchers have taken this leap. Why have they done this? And what are the benefits?

 

1. Task Builders are easier

Using a task builder has a far lower barrier to entry. While coding is fab, and my inner puzzle solving, lego build­ing child self just LOVES tin­ker­ing with code and making it COME ALIVE, I do rec­og­nize that for some people it’s a com­plete turn off.

Syntax, vari­ables, if else state­ments just aren’t everybody’s jam. The same way that – for me – writing is a painful process – for others coding just isn’t their thing.

Task builder set you free from the painful side of coding. When they are done right, you can do things like:

• add content ele­ments (video, image, text) you want on the screen,
• set the content, and make it change trial by trial
• deter­mine how screens should flow from one to the other
• collect a range of inputs (key­board, mouse, voice, video)
• measure reac­tion times, accu­ra­cy, gaze

… And you’re off… All the pain is gone.

And all you’re left with is the ‘it’s alive’ feeling.

Several uni­ver­si­ties are now using task builders as part of their under­grad­u­ate research methods course, so the barrier to entry is low. Not quite as low as eating choco­late, but… you get the idea!

 

2. Task Builders are faster

Build­ing a task in a task builder is unde­ni­ably faster. A simple 4 way forced choice experiment with image stimuli, might only take 15 minutes to create from scratch and deploy online. More time will be spent cre­at­ing the stimuli than cre­at­ing the tech­ni­cal imple­men­ta­tion of the experiment. Which leads me onto my next point…

 

3. Task Builders are cheaper

The econ­o­mist in me likes to price the time saved. Let’s say you’re a post-doc earning £36,000 a year. In code, build­ing your task might have taken you 1 month (£9k) or 2 weeks (£4.5k).

Using a task builder, even if this took you a whole day (£450) – you’re still saving £4,050 in oppor­tu­ni­ty cost.

You can now spend that time design­ing a better study, doing a better analy­sis, writing a better paper, or spend­ing some time relax­ing with friends or family, perhaps evening going to the gym or cooking your­self a deli­cious meal!

 

4. Task Builders are more transparent

Have you ever inher­it­ed someone else’s code? There’s nothing worse, except maybe inher­it­ing someone else’s notes. It’s often incom­pre­hen­si­ble. To be honest, unless it’s very well com­ment­ed, it might even be incom­pre­hen­si­ble to the orig­i­nal author within a suf­fi­cient delay.

Task builders are dif­fer­ent. Because you are working within a frame­work, you can clearly see the design choices that have been made. You may not under­stand the design choices, and you may dis­agree with them. But the choices made are clear.

 

5. Task Builders are more democratic

Not only are they clear to someone with tremen­dous tech­ni­cal skill. They are also clear to someone with very little tech­ni­cal skill. So, task builders are both trans­par­ent and democratic.

 

6. Tasks built in builders last longer

Counter-intu­itive­ly, task builders provide more tech­ni­cal longevi­ty. When you’ve written some­thing in code, it will be inter­pret­ed by the target program (i.e. a browser) exactly as you have specified.

Perhaps you were writing your code back in the 1990s and you spec­i­fied the layout based on the posi­tion (pixels from the top and pixels from the left). When screen res­o­lu­tions increased, your code needs to be updated. In con­trast, there is always a level of inter­pre­ta­tion between a task builder and what the par­tic­i­pant sees. As web stan­dards change (which they will) and hard­ware changes (at a relent­less pace) the provider of your pre­ferred task builder will update how the spec­i­fi­ca­tion is inter­pret­ed to ensure that your task con­tin­ues to run.

 

7. Task builders provide better timing

One crucial ques­tion for behav­iour­al sci­en­tists is which tool will give them the best timing. A good task builder will provide you with the best in class and val­i­dat­ed timing (see timing paper). This is because those cre­at­ing task builders (1) know what they are doing and (2) have the resources to val­i­date their implementation.

In con­trast, while it’s pos­si­ble to get excel­lent timing in code, the respon­si­bil­i­ty falls to the indi­vid­ual and so is con­strained by your tech­ni­cal ability. You may also not have the resources to val­i­date your imple­men­ta­tion as that requires spe­cial­ist hard­ware and is time-consuming.

 

8. Task builders are more efficient

Those that create task builders are highly moti­vat­ed to create a wide range of samples, so that their clients don’t have to rein­vent the wheel. So it might be that your task is already built and ready to go!

Gorilla also pro­vides an open access repos­i­to­ry where researchers share their tasks for others to use. This pro­vides a hugely effi­cient way of encour­ag­ing repli­ca­tions and build­ing on each other’s work.

 

9. Task builders provide spe­cial­ist support

Sim­i­lar­ly, those that create paid for task builders are highly moti­vat­ed to provide excel­lent cus­tomer support.

Second, and more impor­tant­ly, the support team at Gorilla is the best I have worked with in my research career. They respond prompt­ly and thought­ful­ly to resolve any issues along the way in experiment building..

 

MVM, Notre Dame

This is a huge benefit. You have the equiv­a­lent of a spe­cial­ist research assis­tant only an email, phone or video call away. When you get stuck and don’t know what to do, you don’t have to ask someone a favour and wait for them to be free. Instead, there is a team whose job it is to support you.

 

10. Task builder (often) provide a con­sult­ing service

A good task builder will also often offer a con­sult­ing service. We are fre­quent­ly hired by clients to add new func­tion­al­i­ty to Gorilla (eye track­ing, canvas paint­ing, AAT zone, real effort zone). This is also an excel­lent use of grant money as the feature not only ben­e­fits the orig­i­nal research team, but also becomes avail­able to all other researchers using Gorilla around the world, which is cur­rent­ly over 10,000! The return on invest­ment for the research com­mu­ni­ty is tremendous.

So with all these ben­e­fits, why are some people still writing in code?

Are there any down­sides to using task builders?

To answer this ques­tion, it’s useful to step back and think about the devel­op­ment process of a task builder. Those cre­at­ing task builders will be adding UI (user inter­face) to provide func­tion­al­i­ty that is wanted by many people.

The effi­cien­cy comes because the func­tion­al­i­ty is written once by an expert research soft­ware devel­op­er, and used many times by researchers. However, behav­iour­al science exper­i­ments some­times need func­tion­al­i­ty that has never been used before. So, one of the trade-offs is getting exactly what you want through their GUI.

Nev­er­the­less, the best task builders allow you to augment the func­tion­al­i­ty that they provide with your own bespoke code. This gives you the best of both worlds, a UI for aspects of your experiment that aren’t novel, and code for the novel elements.

There are two generic approach­es to aug­men­ta­tion. The first is that you build your task in the GUI, then export it to code, and then make changes. This approach means that you can’t then go back to the GUI to make changes which can be frustrating.

The second, is to build your task in the GUI and then be able to add snip­pets of code that add addi­tion­al func­tion­al­i­ty while staying in the task builder. This latter option is better as you are still able to make changes using the GUI in par­al­lel with tweak­ing your bespoke addi­tions. The latter is how Gorilla works.

But… Learn­ing to code is valuable

Yes it is!

Imagine that you’re an under­grad­u­ate or master’s student in psy­chol­o­gy, or behavioral eco­nom­ics. Given that you don’t need to, is learn­ing to code in JavaScript the best use of your time? Or would it be better spent mas­ter­ing the science and sta­tis­tics com­po­nents of your course? Or perhaps you have other uses for your pre­cious time? Like earning money? Or seeing friends? Or pur­su­ing hobbies? Exer­cis­ing? Meditating?

If you really want to code, your time may be better spent mas­ter­ing data analy­sis in R. Once you are con­fi­dent with the basics of coding move onto aug­ment­ing func­tion­al­i­ty in a task builder. This approach makes sure that you are always getting a good benefit for the time you invest in learn­ing to code

Build­ing and main­tain­ing a plat­form is complex

Build­ing and main­tain­ing a whole plat­form for running online tasks is complex tech­ni­cal­ly and legally. You’d need to:

• write the par­tic­i­pant experience
• set up a data­base and server
• manage your SSL certificate
• write inte­gra­tions with par­tic­i­pant recruit­ment services
• ensure your solu­tion meets your insti­tu­tion’s ethical guide­lines, and have it verified

This might be too much for some to add to their plate.

Avoid this night­mare scenario!

And you def­i­nite­ly want to avoid the horror sce­nario that hap­pened to one of our (now) clients. They spent 3 months cre­at­ing their own server archi­tec­ture out of open-source com­po­nents to host their task. Their ethics board said they couldn’t use it, so they came to us to host it instead.

Ok – So I should use a Task Builder, but which one?

As far as we know, Gorilla is the only experiment builder on the market. Not only do we provide a task builder, we also provide an GUI experiment design tool, which makes it easy to design simple and complex experiment pro­to­cols. Our tool pro­vides ran­dom­iza­tion, branch­ing, coun­ter­bal­anc­ing, as well as repeat and delay nodes for multi-day studies.

We’ve also col­lat­ed a handy guide com­par­ing the dif­fer­ent task builders avail­able. Read our post com­par­ing task builders here.