10 reasons why task builders are better than writing code
10 rea­sons why task builders are bet­ter than writ­ing code

Today many exper­i­men­tal psy­chol­o­gists and other behav­ioral sci­en­tists turn to task and exper­i­ment builders to help them cre­ate exper­i­men­tal pro­to­cols. But many don’t. They still pre­fer to spend the time writ­ing their exper­i­ments in code.

So if you’re still writ­ing code and you’re won­der­ing whether to con­sid­er using a task builder, here are 10 rea­sons why using a task builder might be a good idea.

Why use a task build­ing tool rather than writ­ing code from scratch?

New tools have always pre­ced­ed rapid inno­va­tion, because they make it easy for large num­bers of peo­ple to do some­thing that was pre­vi­ous­ly restrict­ed to a niche expert group.

Give ordi­nary peo­ple the right tools, and they will design and build the most extra­or­di­nary things.

 

Neil Ger­shen­feld

Over the last few years, sev­er­al tasks builders have emerged that can be used by exper­i­men­tal psy­chol­o­gists and econ­o­mists to build behav­iour­al exper­i­ments. For a review of tools avail­able see here.

Many researchers have taken this leap. Why have they done this? And what are the benefits?

1. Task Builders are easier

Using a task builder has a far lower bar­ri­er to entry. While cod­ing is fab, and my inner puz­zle solv­ing, lego build­ing child self just LOVES tin­ker­ing with code and mak­ing it COME ALIVE, I do rec­og­nize that for some peo­ple it’s a com­plete turn off.

Syn­tax, vari­ables, if else state­ments just aren’t everybody’s jam. The same way that – for me – writ­ing is a painful process – for oth­ers cod­ing just isn’t their thing.

Task builder set you free from the painful side of cod­ing. When they are done right, you can do things like:

  • add con­tent ele­ments (video, image, text) you want on the screen,
  • set the con­tent, and make it change trial by trial
  • deter­mine how screens should flow from one to the other
  • col­lect a range of inputs (key­board, mouse, voice, video)
  • mea­sure reac­tion times, accu­ra­cy, gaze

… And you’re off… All the pain is gone.

And all you’re left with is the ‘it’s alive’ feeling.

Sev­er­al uni­ver­si­ties are now using task builders as part of their under­grad­u­ate research meth­ods course, so the bar­ri­er to entry is low. Not quite as low as eat­ing choco­late, but… you get the idea!

2. Task Builders are faster

Build­ing a task in a task builder is unde­ni­ably faster. A sim­ple 4 way forced choice exper­i­ment with image stim­uli, might only take 15 min­utes to cre­ate from scratch and deploy online. More time will be spent cre­at­ing the stim­uli than cre­at­ing the tech­ni­cal imple­men­ta­tion of the exper­i­ment. Which leads me onto my next point…

3. Task Builders are cheaper

The econ­o­mist in me likes to price the time saved. Let’s say you’re a post-doc earn­ing £36,000 a year. In code, build­ing your task might have taken you 1 month (£9k) or 2 weeks (£4.5k).

Using a task builder, even if this took you a whole day (£450) – you’re still sav­ing £4,050 in oppor­tu­ni­ty cost.

You can now spend that time design­ing a bet­ter study, doing a bet­ter analy­sis, writ­ing a bet­ter paper, or spend­ing some time relax­ing with friends or fam­i­ly, per­haps evening going to the gym or cook­ing your­self a deli­cious meal!

4. Task Builders are more transparent

Have you ever inher­it­ed some­one else’s code? There’s noth­ing worse, except maybe inher­it­ing some­one else’s notes. It’s often incom­pre­hen­si­ble. To be hon­est, unless it’s very well com­ment­ed, it might even be incom­pre­hen­si­ble to the orig­i­nal author with­in a suf­fi­cient delay.

Task builders are dif­fer­ent. Because you are work­ing with­in a frame­work, you can clear­ly see the design choic­es that have been made. You may not under­stand the design choic­es, and you may dis­agree with them. But the choic­es made are clear.

5. Task Builders are more democratic

Not only are they clear to some­one with tremen­dous tech­ni­cal skill. They are also clear to some­one with very lit­tle tech­ni­cal skill. So, task builders are both trans­par­ent and democratic.

6. Tasks built in builders last longer

Counter-intu­itive­ly, task builders pro­vide more tech­ni­cal longevi­ty. When you’ve writ­ten some­thing in code, it will be inter­pret­ed by the tar­get pro­gram (i.e. a brows­er) exact­ly as you have specified.

Per­haps you were writ­ing your code back in the 1990s and you spec­i­fied the lay­out based on the posi­tion (pix­els from the top and pix­els from the left). When screen res­o­lu­tions increased, your code needs to be updat­ed. In con­trast, there is always a level of inter­pre­ta­tion between a task builder and what the par­tic­i­pant sees. As web stan­dards change (which they will) and hard­ware changes (at a relent­less pace) the provider of your pre­ferred task builder will update how the spec­i­fi­ca­tion is inter­pret­ed to ensure that your task con­tin­ues to run.

7. Task builders pro­vide bet­ter timing

One cru­cial ques­tion for behav­iour­al sci­en­tists is which tool will give them the best tim­ing. A good task builder will pro­vide you with the best in class and val­i­dat­ed tim­ing (see tim­ing paper). This is because those cre­at­ing task builders (1) know what they are doing and (2) have the resources to val­i­date their implementation.

In con­trast, while it’s pos­si­ble to get excel­lent tim­ing in code, the respon­si­bil­i­ty falls to the indi­vid­ual and so is con­strained by your tech­ni­cal abil­i­ty. You may also not have the resources to val­i­date your imple­men­ta­tion as that requires spe­cial­ist hard­ware and is time-consuming.

8. Task builders are more efficient

Those that cre­ate task builders are high­ly moti­vat­ed to cre­ate a wide range of sam­ples, so that their clients don’t have to rein­vent the wheel. So it might be that your task is already built and ready to go!

Gorilla also pro­vides an open access repos­i­to­ry where researchers share their tasks for oth­ers to use. This pro­vides a huge­ly effi­cient way of encour­ag­ing repli­ca­tions and build­ing on each other’s work.

9. Task builders pro­vide spe­cial­ist support

Sim­i­lar­ly, those that cre­ate paid for task builders are high­ly moti­vat­ed to pro­vide excel­lent cus­tomer support.

Sec­ond, and more impor­tant­ly, the sup­port team at Gorilla is the best I have worked with in my research career. They respond prompt­ly and thought­ful­ly to resolve any issues along the way in exper­i­ment building..

 

MVM, Notre Dame

This is a huge ben­e­fit. You have the equiv­a­lent of a spe­cial­ist research assis­tant only an email, phone or video call away. When you get stuck and don’t know what to do, you don’t have to ask some­one a favour and wait for them to be free. Instead, there is a team whose job it is to sup­port you.

10. Task builder (often) pro­vide a con­sult­ing service

A good task builder will also often offer a con­sult­ing ser­vice. We are fre­quent­ly hired by clients to add new func­tion­al­i­ty to Gorilla (eye track­ing, can­vas paint­ing, AAT zone, real effort zone). This is also an excel­lent use of grant money as the fea­ture not only ben­e­fits the orig­i­nal research team, but also becomes avail­able to all other researchers using Gorilla around the world, which is cur­rent­ly over 10,000! The return on invest­ment for the research com­mu­ni­ty is tremendous.

So with all these ben­e­fits, why are some peo­ple still writ­ing in code?

Are there any down­sides to using task builders?

To answer this ques­tion, it’s use­ful to step back and think about the devel­op­ment process of a task builder. Those cre­at­ing task builders will be adding UI (user inter­face) to pro­vide func­tion­al­i­ty that is want­ed by many people.

The effi­cien­cy comes because the func­tion­al­i­ty is writ­ten once by an expert research soft­ware devel­op­er, and used many times by researchers. How­ev­er, behav­iour­al sci­ence exper­i­ments some­times need func­tion­al­i­ty that has never been used before. So, one of the trade-offs is get­ting exact­ly what you want through their GUI.

Nev­er­the­less, the best task builders allow you to aug­ment the func­tion­al­i­ty that they pro­vide with your own bespoke code. This gives you the best of both worlds, a UI for aspects of your exper­i­ment that aren’t novel, and code for the novel elements.

There are two gener­ic approach­es to aug­men­ta­tion. The first is that you build your task in the GUI, then export it to code, and then make changes. This approach means that you can’t then go back to the GUI to make changes which can be frustrating.

The sec­ond, is to build your task in the GUI and then be able to add snip­pets of code that add addi­tion­al func­tion­al­i­ty while stay­ing in the task builder. This lat­ter option is bet­ter as you are still able to make changes using the GUI in par­al­lel with tweak­ing your bespoke addi­tions. The lat­ter is how Gorilla works.

But… Learn­ing to code is valuable

Yes it is!

Imag­ine that you’re an under­grad­u­ate or master’s stu­dent in psy­chol­o­gy, or behav­ioral eco­nom­ics. Given that you don’t need to, is learn­ing to code in JavaScript the best use of your time? Or would it be bet­ter spent mas­ter­ing the sci­ence and sta­tis­tics com­po­nents of your course? Or per­haps you have other uses for your pre­cious time? Like earn­ing money? Or see­ing friends? Or pur­su­ing hob­bies? Exer­cis­ing? Meditating?

If you real­ly want to code, your time may be bet­ter spent mas­ter­ing data analy­sis in R. Once you are con­fi­dent with the basics of cod­ing move onto aug­ment­ing func­tion­al­i­ty in a task builder. This approach makes sure that you are always get­ting a good ben­e­fit for the time you invest in learn­ing to code

Build­ing and main­tain­ing a plat­form is complex

Build­ing and main­tain­ing a whole plat­form for run­ning online tasks is com­plex tech­ni­cal­ly and legal­ly. You’d need to:

  • write the par­tic­i­pant experience
  • set up a data­base and server
  • man­age your SSL certificate
  • write inte­gra­tions with par­tic­i­pant recruit­ment services
  • ensure your solu­tion meets your insti­tu­tion’s eth­i­cal guide­lines, and have it verified

This might be too much for some to add to their plate.

Avoid this night­mare scenario!

And you def­i­nite­ly want to avoid the hor­ror sce­nario that hap­pened to one of our (now) clients. They spent 3 months cre­at­ing their own serv­er archi­tec­ture out of open-source com­po­nents to host their task. Their ethics board said they couldn’t use it, so they came to us to host it instead.

Ok – So I should use a Task Builder, but which one?

As far as we know, Gorilla is the only exper­i­ment builder on the mar­ket. Not only do we pro­vide a task builder, we also pro­vide an GUI exper­i­ment design tool, which makes it easy to design sim­ple and com­plex exper­i­ment pro­to­cols. Our tool pro­vides ran­dom­iza­tion, branch­ing, coun­ter­bal­anc­ing, as well as repeat and delay nodes for multi-day studies.

We’ve also col­lat­ed a handy guide com­par­ing the dif­fer­ent task builders avail­able. Read our post com­par­ing task builders here.