Academic researchers
Academic researchers
We can all do our bit to increase the reliability of researchOne small step at a time
Open and reproducible science initiatives are being introduced by the scientific community to increase the transparency and reproducibility of research.
In recent years, concerns have been raised that the lack of transparency, reproducibility, and the pressure to publish are negatively impacting on biomedical science research such as neuroscience. In response to what is referred to by some as the ‘Reproducibility Crisis’, the psychology community in particular has been trialling various ‘Open Science’ and ‘Reproducible Science’ solutions. Whilst growing in popularity among psychologists, these Open/Reproducible initiatives are relatively new concepts to broader neuroscience.
These initiatives include, for example: Open data, materials and code/analysis, Preregistration, Registered Reports, and Reproducibility/Reanalysis & Replication studies. Descriptions of these terms (and more) can be found in the glossary. Understandably, there are questions - and hopefully, we have covered many of these in the toolkit
and FAQ
sections.
Below we suggest ways you can start introducing open and reproducible science initiatives into your work.
Undergraduate students
We are in an exciting era of science research.
As technology advances, as does the complexity of our experiments, our ability to obtain large volumes of data, and the sophistication of data analysis & modelling methods.
As exciting as technology advancements are, it has never been more important to make sure that our studies are well designed, and our research questions well defined.
Sharing our study design with the wider research community helps generate advice and feedback at this crucial stage of the scientific process.
An easy way to openly share your study design is by study preregistration.
Preregistration is most commonly performed by posting a research plan on an independent online registry e.g., Open Science Framework
(and from here can be shared with the wider research community).
More information on Study preregistration can be found in the toolkit and at the Center for Open science.
Do one thing - learn about study preregistration
Postgraduates
Presenting your work at a conference is a valuable experience. It is an opportunity to talk about your research and pick up advice and feedback from the research community. However, wouldn't it be great if you were able to present and obtain feedback on your study design, rather than on work that has already been completed?
Preregistration posters
At the BNA2019
festival of neuroscience we introduced preregistration posters. For this, a preregistration poster had to include the following elements:
- Introduction
- Methods
- Approach for statistical analysis
This new poster format allows neuroscientists to obtain feedback (and learn) from the international neuroscience community when it is most valuable (when there was is chance to revise the plan) and form collaborations that will help with the proposed study. Presenters of preregistration posters are also awarded a Preregistration Open Science badge.
More information on preregistration posters can be found in the toolkit.
We have also developed a prereg poster badge
you can use for your poster.
To find out which conferences currently accept preregistration posters, please see the COS website
(under the Allied Initiatives tab).
Do one thing - present a preregistration poster
Postdoctoral researchers
The research question and methodology were sound and the experiments were conducted as planned. But the results were negative/insignificant. Now what?
In our conventional publishing system, preference is shown for positive/significant, novel and high impact data. But all data resulting from robust and well conducted studies (positive, negative or otherwise) should be published irrelevant of outcome.
Journals are increasingly aware of the role they play in increasing the credibility of research and are creating formats that encourage open reporting.
For example, at our journal, Brain and Neuroscience Advances, we publish null results
from robust studies that fail to reject the null hypothesis.
Registered reports
We have also introduced the Registered Reports
format at our journal. Unlike traditional articles, Registered reports
are accepted for publication at the study design stage, prior to work being carried out. This means we will publish your work regardless of the outcome (and be awarded a Preregistration Open Science badge).
More information on Registered Reports can be found in the toolkit
and at the Center for Open science.
Do one thing - Publish negative results
Fellows and Principal Investigators
Adoption of open and reproducible research practices may seem impossible in the reality of our current research culture - not to mention the lack of incentives, support and infrastructure.
But things are changing, and many funders and research councils are committed to changing the way they evaluate research and researchers. And along with a growing number of institutions, they are beginning to provide the appropropriate infrastructure and training.
First to change: transparency
Increasingly, funders and journals are requesting that all methods, materials, data and code associated with a project and/or published article be openly shared via an online repository.
Project management tools
To help streamline the transition of information from lab to online repositories, it may useful to start introducing into your lab/group to digital data/project management tools. For example:
- The Open Science Framework (OSF) is a free, open source, web-based project management tool offered by the Center for Open Science. The framework facilitates the managing and sharing of research.
- Electronic lab books integrate experimental notes (and in some cases experimental software) with data collection/data management (an introduction to electronic notebooks can be found here).
- Investigate whether your institution already offers a service for organising and archiving data and study materials.
Do one thing - digitalise study workflows