In this guest blog we hear from Dr James Bartlett about the importance of researchers sharing their research materials online. Thanks James!

When we teach students about writing an effective method section, we normally emphasise providing enough detail to allow another researcher to replicate the study. In reality, it can end up resembling a botched restoration of a 16th century artwork. You’re aiming for the original image, but in the end, you can only see the resemblance if you squint and tilt your head to one side.

Left image: Ecce Homo by Elias Garcia Martinez. Right image: Restoration attempt by Cecilia Giménez.

As rigorous scientists, our desire is to run an exact replication, but frustratingly the details are rarely specific enough to allow it. This is a problem, as when researchers try to replicate the study without the original stimuli, inconsistencies in the results can be attributed to differences in the materials or setting (Klein et al., 2018). In an ideal world, the exact materials would accompany the article to provide an independent researcher all they need to recreate the study. However, as we will see, sharing materials is rare. After hopefully convincing you why this should change, this article will provide a brief overview of options and considerations for sharing your materials.

Sharing is rare but rewarding

Researchers seldom share the materials used in their experiment, with studies auditing the prevalence of open materials ranging from 1% (Adewumi et al., 2021) to 14% of articles (Hardwicke et al., 2021). Even looking for a lower bar of transparency, just 38% of articles reported where they sourced the images in their cognitive tasks (Pennington et al., 2021). These studies show disappointingly low rates of transparency in materials across areas of psychology. As different areas of science have worked to become more open, being transparent about how you conducted your study is one of the key tenets (Munafó et al., 2017). Despite this, in a world where the quantity not quality of research is valued (Allen & Mehler, 2019), going out of your way to document and share the materials is typically not high on the agenda. What is the incentive? One initiative is journals providing badges to recognise articles that have shared outputs like the materials. In the journal Psychological Science, offering badges for open materials increased rates from 13% to 30%, while control journals not offering these badges remained stable (Kidwell et al., 2016). More journals have offered these badges (Grahe, 2018), but uptake is still the exception, not the norm. Perhaps more enticing is articles providing open materials tend to receive a higher number of citations (McKiernan et al., 2016). These initiatives show that beyond the warm fuzzy feeling of being helpful, researchers can be recognised for sharing their materials through a badge or the prospect of more citations. Effectively sharing and documenting materials takes extra time and can present a barrier for researchers as they are perceived to not be valued in the race for funding and jobs (Zečević et al., 2021). However, sharing materials is often one of the easiest open science practices to adopt as it does not have to be in the ethics process like open data. This means if you want to try ways of making your research more open and transparent, sharing your materials is one of the easiest starting points.

Gorilla Open Materials

In my teaching and research, I use Gorilla to create experiments as I work for an online university. One of their features is an Open Materials function. This allows you to share and document any task, questionnaire, code, or even whole experiment. For example, in my PhD I used something called the visual probe task with images of smoking/cigarettes. I added it to the Open Materials page:

Gorilla Open Materials overcomes many of the barriers to sharing materials. All you have to do is add a task you already made to an Open Materials page. You provide a description of the task and what it does. Further below in the screenshot, you can add a task diagram to provide a visual overview. For those sweet citation counts, you can add your preferred citation in the top right. The most important aspect is the ease of reuse. There are preview and clone buttons with each task. Previewing the task allows anyone to experience it exactly as the participant did in your browser. This means anyone can see exactly how it worked, without downloading any software or signing up for a Gorilla account. If a researcher wants to inspect, reuse or adapt it in their own work, they can clone the task into their own project. If you use Gorilla for your research, this presents an extremely user-friendly way of facilitating replications of your work.

Note from the Gorilla team: We’ve heard that some researchers include their Gorilla Open Materials page within pre-registrations on the Open Science Framework which then also provides a DOI for the resources. In exciting news — Gorilla Open Materials 2.0 will have the ability to add DOIs! Keep your eyes peeled for more information on this…

Sharing task scripts on the OSF

If you do not use Gorilla, you can share your task scripts on a repository to allow another researcher to reuse it. For example, if you have created an experiment using E‑Prime or Matlab, you can share the script and stimuli used to program the task. If these are all you have, then sharing is better than not sharing, but these are both examples of propriety software. This means someone would not be able to use the materials until they bought a licence. If you share scripts from open-source alternatives, anyone in the world can reuse them. For example, I have shared the scripts from OpenSesame and PsychoPy on the Open Science Framework:

For example, this screenshot shows a zip file containing all the PsychoPy files needed to run the visual probe task. The only addition I had to make was writing a README file which describes how it can be reused. It describes the trial structure, how to replace the stimuli, and how to use the task without the eye-tracking code. I know at least three people have benefited from sharing these scripts on the OSF, as they have emailed me to ask questions. With OSF metric tracking, I can also see this task has been downloaded 220 times at the time of writing. This shows how a small time commitment can benefit other researchers. In addition to pure communalism, the OSF project has a DOI which means other researchers can cite the project when they reuse it. This shows you can also benefit from sharing your materials as it contributes to your impact.

Sharing questionnaires

Not all research is behavioural, therefore consideration of questionnaire access is important. When a team of researchers validates and publishes a questionnaire, the items and scoring instructions can often be missing. Sometimes this will be due to concerns over the journal retaining copyright, while others could create proprietary questionnaires and require payment for reuse (Hays et al., 2018). If you validate a questionnaire and want to share the materials for maximum reuse, you can release it under a Creative Commons licence. This means you can openly share the questionnaire while still retaining rights as the author (Hays et al., 2018). Before you submit the validation article, you could add the questionnaire to Gorilla Open Materials, the OSF, or Figshare. The OSF and Figshare allow you to create a DOI, select a licence to state terms of reuse, and if required, set an embargo period until there is public access. This means you can cite the questionnaire in your article and retain rights as the author and encourage free use as long as you are cited as the original.

Note from the Gorilla Team: Within Gorilla Open Materials there are three licences to choose from:

• Creative Commons Attribution
• Creative Commons Attribution Non-Commercial
• Gorilla Open Materials Attribution Non-Commercial Research Only

The third licence is only available in Gorilla. It’s for situations where you may have a measure (such as an IQ test) and you neither want it to be used commercially or in the public sector (i.e., schools and hospitals). It only allows others to use it for non-commercial research purposes.

Sharing a video of the materials and setting

Sharing the materials is important, but an additional concern in replication studies are differences in the settings and procedure (Klein et al., 2018). One initiative some researchers have started is sharing a recording documenting the data collection procedure. For example, Wagenmakers et al. (2016) shared the materials from their replication attempt and a recording of the procedure. This makes the method crystal clear: making it easier to reuse the materials and reduces the opportunity for differences in procedure to affect the meaning of failures to replicate.

Conclusion

The movement towards greater transparency in research means sharing materials has never been easier with tools like Gorilla Open Materials, the OSF, and Figshare. However, estimates in psychology of how often researchers share their methods are disappointingly low given how it is one of the easiest parts of the research process to open up. Although it takes extra time, you can be rewarded with the warm fuzzy feeling of helping other researchers, and if that does not motivate you, the potential for a greater number of citations and badges to reward your effort might help. There are different options available for sharing your materials, from Gorilla Open Materials, to sharing your tasks and questionnaires on the OSF. Just remember to make it reusable for other researchers by documenting how it works through a README file. Hopefully, you will see the rewards of greater transparency from researchers being able to reuse and credit your materials as an additional output, to when you need to replicate a study and you find the authors helpfully posted a video demonstrating their procedure.

Dr James Bartlett

Dr James Bartlett (@JamesEBartlett), Senior Lecturer, School of Psychology and Social Science, Arden University, UK.