Neuroimaging research

Neuroimaging toolkit

Easy to implement tips on how to make your neuroimaging research as credible as possible


If details on methods and key details are not available to other researchers, this can be a barrier to them reproducing experiments, building on them, or considering how reliable the results are likely to be. Concerns have been raised that the conclusions drawn from some human neuroimaging studies are either spurious or not generalisable, and that low statistical power, flexibility in data analysis, and software errors can impact the credibility of these studies [1]. 


This toolkit aims to give neuroscientists conducting neuroimaging research some of the key ways to strengthen credibility of their work that they should aim to do, and which are simple to implement, at the stages you plan your research, carry the study out, and after you've analysed your data. 

Planning your study

1. Preregister your study

You can reduce bias and build confidence in the study by preregistering a time-stamped account of your research plans prior to starting the study, clearly and openly stating your experimental rational, hypothesis, and methods, including the intended statistical analysis. This helps to demonstrate not only the rigorous planning, but also that these aspects of the study were not shaped during or after data collection and that the results are not selectively reported. 


The easiest way to preregister a study is via a registry. There are all-purpose registries for a range of scientific disciplines, such as the Open Science Framework.  

 

You can also preregister a study through a journal that offers Registered Reports, a publishing format that incorporates preregistration. This is offered through, for example, the BNA's Journal Brain and Neuroscience Advances. To find out more, check out our toolkit on preregistration and Registered Reports.

 

To aid you in writing your preregistration you can use templates. Most templates will include lots of bullet points that you will need to address and specify all the parameters of your study. You can use templates that are tailored to specific research fields. For example within the field of neuroimaging here are templates for an FMRI study or for an EEG ERP study. Watch Dr Roni Tibon from the University of Nottingham explain the preregistration template options below (excerpt from a BNA webinar on preregistering neuroimaging studies). 

2. Sample size and future planning

The most common issues you will face when writing your preregistration are deciding a suitable sample size, controlling subjective aspects, and accommodating for additional analysis. Reproducible brain associations can require thousands of individuals [2], yet some of the most highly cited studies have low sample sizes and a very small number of studies analysed have power calculations [3]. 

 

When conducting neuroimaging research, it can be difficult to decide on a realistic and feasible sample. Whilst a larger sample makes your research more reliable, neuroimaging is expensive and time consuming, this limits the number of participants you can recruit. Sample size might also depend on the specific design - for example, task-based studies normally require smaller samples than resting-state studies [4]. To help you decide, you can review previous literature as an aid. Where this is hard to estimate, simulations may be useful in determining sample size for different effect sizes (e.g. MRC CBU have developed a simulation tool using Bayesian sequential design). 


To help with power calculations, there are resources online for use in fMRI research, such as fMRIpower and a power calculation guide for fMRI. 


Unfortunately, some aspects of research will be subjective such as any manual steps. There are community efforts to try and remove subjective elements however these procedures may reduce a general understanding of the data. As long as all manual steps are applied in a way that is blind to the experimental conditions and the data quality is estimated using contrast that is orthogonal to the contrast of interest, then you can still include them. 

 

Analysis can be registered or exploratory. Exploratory research can lead to many levels of analysis and new hypotheses. As long as you still address your original hypothesis, you can still complete additional analyses.  

3. Consider GDPR ahead of time 

It is important to consider before conducting your research if you will want to share your data. Under GDPR there are only two types of data: personal or anonymous. A simple way to reduce the risk of identification is to use pseudonyms, so that you can still share personal data. For data to be considered anonymous it must be irreversible. Neuroimaging data can reveal health concerns that participants should be informed off, therefore when conducting anonymised neuroimaging data often one keeps a key that can reverse the ID of participants. This means the data cannot be entirely anonymised.


When deciding if your data is anonymised, you should consider: 

  • Is it possible to single out an individual? 
  • Is it still possible to link records relating to an individual? 
  • Can information be inferred concerning an individual? 


However, GDPR article 5 states: “The free movement of personal data within the union is not restricted or prohibited for reasons connected with the protection of natural persons with regard to the processing of personal data” [5]. This means if you do discover important medical information during your experiment, you can disclose this to the individual immediately and share the scans to them.


Additionally, as long as a simple data user agreement is agreed with the participants, then the data can be shared with other researchers. The data user agreement, must include participants consent for their data to be used in certain areas of scientific research, if you want to allow other labs to be able to use it. 


For more, check out Cyril Pernet's webinar for the BNA and a recent article in Lancet Digital Health on sharing neuroimaging data.

4. Have a plan for managing the data you collect 

There are simple steps that you can take to improve the structure of how you manage data collection throughout your study. This can not only help you further down the line when it comes to remembering the processes you used, but also provides transparency on how your raw data are turned into your final results. Retaining all raw data is also useful in case at a latter point you discover a error in the data processing or analysis that you need to trace back to the original data. 

 

One tool we can use for structuring our neuroimaging data is the Brain Imaging Data Structure (BIDS). BIDS is a system of organising neuroimaging data, designed to be a robust and reproducible central place for people to analyse and manage their data. BIDS creates simple yet specific structures that are also flexible for all types of neuroimaging, making it easier to organise your data – examples of the BIDS standard are available to view on GitHub  to help you organise your data as best as possible and assert integrity.  

 

Additionally, if you do not wish to upload your data to BIDS manually you can use tools to convert your folders into BIDS format (please note: these convertors are no longer under active development and do not support recent extensions to BIDS standards). dcm2bids is one such tool that reorganises NIfTI files using dcm2niix into BIDS format. For other tools and modalities, check out: https://bids.neuroimaging.io/benefits


If you use BIDS to organise your data appropriately, you can store and share your data using NeuroVault (https://neurovault.org/). It should be noted that you should always keep multiple versions of your data saved to prevent losing it.


The benefits of having a robust and consistent structure are you can use other tools to query and analyse different databases and view data without having to download it. In the clip below from our Credibility Lunchbox webinar series, Etienne Roesch highlights a number of tools that can be used.  

After data collection

5. Make sure data you share is GDPR-compliant 

Before you can share your data you must check it is ok to do so according to GDPR guidelines. If you are sharing personal data (including neuroimages) then you will need: 

  • Participant consent for your study 
  • Participant consent to share data 
  • Institutional approval 


You can use templates such as https://open-brain-consent.readthedocs.io/ to help you follow GDPR principles of lawfulness, transparency and fairness.  

6. Follow FAIR principles for sharing your data 

You should make sure any data you’re intending to share follows the FAIR principles: findability, accessibility, interoperability, and reusability [6]. Data can still be shared in an indexed repository as long as access is controlled so that people can sign the data user agreement. 

 

To be FAIR, sharing personal identifiable information requires consent to share but also an infrastructure allowing public data sharing with access control to identified users using legal agreement(s) such as a Data User Agreement (DUA) and Standard contractual clauses (SCC) of non-EU users. It is best to identify if this is possible early on. Check with your library/institution what is planned, as it is their job to provide infrastructure to support research/funders requirement. You can also consider: 
https://www.healthdatacloud.eu/ https://openneuro.org/


7. Report your results

It's important for the transparency of neuroimaging research that there is accurate and thorough reporting of results. This will not only help others to fully interpret the results, but also be aware of the full set of analyses made from the data to prevent selective reporting. The COBIDAS guidance provides a useful reporting checklist (Appendix D) to help researchers ensure best practice.


There is also potential to upload activation maps (thresholded/unthresholded, subject-level/group-level) onto portals such as NeuroVault for others to browse. This also allows data to be used fully for mega-analyses and meta-analyses. 


8. Share your data and code

Sharing your data via an online repository can be beneficial to both yourself and to science more widely. It ensures that your datasets are secure and accessible in the long term, and gives the dataset a DOI, allowing the data to be cited by others. Demonstrating data-sharing can also be useful when applying to future grants. Making the data available online means these could also in the future be used for further study by others, for example through combining different datasets. 


Sharing code can further strengthen reproducibility by transparently providing the information needed for researchers to repeat analyses of the original data and opening it up to others to highlight any coding errors made [7]. Online platforms such as GitHub and bitbucket can host your code, which can then be included as an open material in your published research.


Useful resources

Andy’s Brain Book: https://andysbrainbook.readthedocs.io/en/latest/


COBIDAS guidance. Nichols TE et al. Best Practices in Data Analysis and Sharing in Neuroimaging using MRI. 2016. bioRxiv doi: 10.1101/054262


FSL documentation: https://fsl.fmrib.ox.ac.uk/fsl/docs/#/


MRC CBU List of openly available datasets: https://imaging.mrc-cbu.cam.ac.uk/methods/OpenDatasets


Niso G et al. Open and reproducible neuroimaging: From study inception to publication. NeuroImage, 263, 1–19. doi: 10.1016/j.neuroimage.2022.119623 Juypter book available at https://oreoni.github.io/index.html


SPM documentation: https://www.fil.ion.ucl.ac.uk/spm/docs/


White T, Blok E, Calhoun VD. Data sharing and privacy issues in neuroimaging research: Opportunities, obstacles, challenges, and monsters under the bed. Hum Brain Mapp. 2022 Jan;43(1):278-291. doi: 10.1002/hbm.25120.


Video: For more useful tips, check out the three short BNA Credibility Lunchbox webinars on neuroimaging neuroscience, available in our recordings section with presenters' slides, or as a single playlist via the BNA's YouTube channel. You can also watch a talk by past BNA President Prof Rik Henson on 'Open Cognitive Neuroscience' that has a number of neuroimaging tips mentioned.


Top


  1. Poldrack R, Baker C, Durnez J et al. Scanning the horizon: towards transparent and reproducible neuroimaging research. Nat Rev Neurosci 18, 115–126 (2017). https://doi.org/10.1038/nrn.2016.167 
  2. Marek S et al. Reproducible brain-wide association studies require thousands of individuals. Nature, 603(7902), 654-660 (2022).
  3. Szucs D & Ioannidis JP. Sample size evolution in neuroimaging research: An evaluation of highly-cited studies (1990–2012) and of latest practices (2017–2018) in high-impact journals. NeuroImage, 221, 11716 (2020).
  4. Tibon, R., Geerligs, L., & Campbell, K. (2022). Bridging the big (data) gap: Levels of control in small- and large-scale cognitive neuroscience research. Trends in Neurosciences, 45(7), 507–516. https://doi.org/10.1016/j.tins.2022.03.011
  5. Art. 5 GDPR – principles relating to processing of personal data (2021) General Data Protection Regulation (GDPR). Available at: https://gdpr-info.eu/art-5-gdpr/ (Accessed: 18 June 2024). 
  6. What is fair? INCF. Available at: https://www.incf.org/what-is-fair (Accessed: 18 June 2024). 
  7. Sharma NK et al. Analytical code sharing practices in biomedical research. bioRxiv [Preprint]. 2023 Aug 7:2023.07.31.551384. https://doi.org/10.1101/2023.07.31.551384.


This toolkit was produced in 2024 by Grace Cooper (BNA Placement Student) with guidance from Olivia Kowalczyk (KCL), Roni Tibon & Josefina Weinerova (University of Nottingham) and Esther Walton (University of Bath).

Share by: