• Student Researcher Credibility Prize winner: Josefina Weinerova (University of Nottingham)  Highly Commended: Brian Schilder (Imperial College London)
• Individual Researcher Credibility Prize winner: Lei Zhang (University of Birmingham)
• Team Credibility Prize winner: CODECHECK

To address these structural issues, a team in academia and industry has launched an effort to systematically characterize widely used antibodies, employing knockout cells and tissues as controls whenever possible. Known as YCharOS, this public-private partnership has initially focused on antibodies used to study neurological conditions, including Alzheimer’s disease, ALS, autism, frontotemporal dementia and Parkinson’s disease. 

DOI: 10.53053/ENUE9175

Four groups in the field of experimental psychology have successfully replicated each other’s work by following best practices. This new paper by Protzko J, Krosnick J, Nelson L et al reports an investigation by four coordinated laboratories of the prospective replicability of 16 novel experimental findings using rigour-enhancing practices: confirmatory tests, large sample sizes, preregistration and methodological transparency. 

Whilst past systematic replication efforts have reported replication rates averaging 50%, replication attempts here produced the expected effects in 86% of attempts. When one lab attempted to replicate an effect discovered by another lab, the effect size in the replications was 97% that in the original study. This high replication rate justifies confidence in rigour-enhancing methods to increase the replicability of new discoveries.

DOI: 10.1038/s41562-023-01749-9

The group of influential funding agencies, cOAlition S, has outlined a future “community-based” and “scholar-led” open-research communication system in which publishers are no longer gatekeepers that reject submitted work or determine first publication dates. Instead, authors would decide when and where to publish the initial accounts of their findings, both before and after peer review. Publishers would become service providers, paid to conduct processes such as copy-editing, typesetting and handling manuscript submissions. cOAlition S is launching a six-month process, co-led by Research Consulting, to collect feedback from members of the global research community on whether the plan will meet their needs.

Traditional peer review, where reviewers know authors’ identities but are not revealed to them, often results in biases related to factors such as the author’s institutional affiliation, gender or reputation, hindering objective evaluation of submissions. In this blog during Peer Review Week, IOP Publishing's Kim Eggleton highlights the growing evidence in support of innovation in peer review, sharing data on the Institute of Physics moving their journal to double-blind peer review, and highlighting improvements that can be made in a more transparent and open process. 

Following our open call for new members of the BNA's Credibility Advisory Board, three new members have accepted invitations to join:

• Olivia Kowalczyk (King's College London)
• Llywd Orton (Manchester Metropolitan University)
• Faisal Mushtaq (University of Leeds)

Key opinion leaders from across the preclinical neuroscience research community have co-developed new guidance for researchers conducting preclinical psychopharmacology studies which include the use of animal models.

Their recent paper, ‘iTRiPP: Improving Translational Relevance in Preclinical Psychopharmacology’, published earlier this month in the Journal of Psychopharmacology, outlines key considerations for the planning and reporting of behavioural studies relevant to psychiatric conditions. The iTRIPP guidelines are intended to complement existing recommendations for planning and reporting animal studies, by enabling researchers to fully consider the most appropriate animal model for the research purpose and to interpret their findings accordingly. 

In a new briefing paper 'Towards better science: More innovative and robust academic research', the Social Market Foundation think tank offers a set of recommendations focused on addressing the reproducibility challenge. It suggests that creating a shared repository, accessible by all researchers in the UK, would make it much easier to access data, code, and pre-prints from other institutions. It also calls for the creation of a new National Institute for Scientific Replicability, to be tasked with ‘red-teaming' science – mimicking the practice from cyber-security of intentionally seeking out vulnerabilities and testing them. This would would sponsor replication projects and develop tools to predict replicability.

The UK Government and UK Research and Innovation (UKRI) have published a joint response to the House of Commons Science, Innovation and Technology (SIT) Committee report on reproducibility. In the response, the UK Government and UKRI partially accept the majority of the conclusions and recommendations from the SIT Committee. 

The four UK higher education funding bodies have published the key decisions following the Future Research Assessment Programme. Changes for REF 2028 include an expansion of the definition of research excellence to ensure appropriate recognition is given to the people, culture and environments that underpin a vibrant and sustainable UK research system. A consultation on further guidance and criteria is open until 6 October. 

The House of Commons Science, Innovation and Technology (SIT) Committee has published a report following its inquiry into the reproducibility of scientific research. The report calls for reforms in the research sector to help tackle the problem of reproducibility, and takes on board a number of issues the BNA included in our written evidence to the inquiry.

At the BNA's 2023 Festival of Neuroscience in Brighton, for the first time prizes were available in an open science category. Abstract submitters were encouraged to apply for Open Science Badges to add to these posters, and posters with these badges were considered by our judging panel: Kate Baker, co-editor of the BNA's Journal, and from the BNA's Credibility Advisory Board, Mike Ashby and Ana Dorrego-Rivas. The winners were:

Early Career Researcher category:

• Winner: Pierce Mullen, University of St Andrews  - "A deep learning toolbox for decoding behaviour in larval zebrafish"
• Runner-up: Petar Raykov, University of Cambridge - "Effects of APOE Genotype on Brain Activity During Movie Watching"

Student category:

• Winner: Heather McCourty, University of Sheffield - "Automated Optimisation of Stem Cell Differentiation into Neurons"
• Runner-up: Kenneth Steel, University of Bristol - "Multiplex Spinal Somatosensory-Evoked Potentials as a High Content Biomarker of Analgesic Actions in Rats"

In a huge boost to the Registered Reports format, Nature has announced that from this week it is offering Registered Reports in the field of cognitive neuroscience and in the behavioural and social sciences. In the future, the publisher plans to extend this to other fields, as well as to other types of study, such as more exploratory research.

Nature is recognising Registered Reports as a progression towards greater emphasis on rigour and study design, and that the format strengthens both how science is done and the notion that good research starts well before the paper is written.

The winners of the 2023 BNA Credibility Prizes have today been announced. 

Student Researcher Prize
Winner: Maria Korochkina (Royal Holloway, University of London)  

Highly Commended: Abigail Fiske (University of Oxford)

 

Individual Researcher Prize
Winner: Roni Tibon (University of Nottingham)

Highly Commended: Esther Walton (University of Bath)

 

Team Prize
Winner: #EEGManyLabs Project Team

Highly Commended: Dementias Platform UK

In this guest piece, Erika Pastrana (Editorial Director for the Nature Portfolio and Open Science lead at Springer Nature) and Simon Adar (CEO and co-founder of Code Ocean) consider some of the barriers to Open Science.  They suggest that where open research practices are taking place more consistently, integrated offerings seem to be available to researchers as part of the article submission process.

In this article, Marcus Munafò and Neil Jacobs from the UK Reproducibility Network describe how UKRN's Open Research Programme is helping institutions rise to the challenge of open research, and highlight how investment, training and incentives are required if the sector is going to truly embrace it. 

Alzheimer's Research UK (ARUK) today launched its new five-year research strategy, developed by its Scientific Advisory Board and with input from people affected by dementia. It identifies and addresses the major opportunities for progress in dementia research over the coming years.

The strategy sets out three priority areas to make a difference to people’s lives: treatment, diagnosis and prevention. As part of fostering a research culture that improves translation, the new strategy also states that ARUK "will define and develop a programme to address issues with reproducibility in research".

In this article, Randall Ellis from the Friedman Brain Institute in New York analyses the literature to highlight the contributing factors influencing replicability, such questionable research practices, misunderstanding of p-values, and low statistical power, and how these issues manifest specifically in preclinical neuroscience research. 

While the problems are multifaceted and difficult to solve, the article argues that changing academic incentives are key to strengthening replicability, with adoption of registered reports being the most immediately impactful and pragmatic strategy. For in vivo research in particular, comprehensive reporting guidelines are an essential addition to this.

A new study has found that biomedical and health researchers who declare they are happy to share the data behind journal articles do not respond to access requests or hand over the data when asked.

A team at the Catholic University of Croatia in Zagreb analysed 3,556 biomedical and health science articles published in a month by 282 BMC journals, identifying 381 articles with links to data stored in online repositories and another 1,792 papers for which the authors indicated in statements that their data sets would be available on reasonable request. Of these, more than 90% of corresponding authors either declined or did not respond to requests for raw data. 

The BNA has submitted its response to the Future Research Assessment Programme, which is considering what might replace the Research Excellence Framework in the future.

Around 100 neuroscientists contributed to our response following a survey the BNA conducted seeking their views on a future assessment system. We have recommended that institutions are incentivised to demonstrate how they are helping to provide support for managing and curating the research outputs produced.

The winners of the 2022 BNA Credibility Prizes have today been announced. 

Student Researcher Prize
Winner: Fiona Ramage (University of Dundee)  

Highly Commended: Niamh MacSweeney (University of Edinburgh)

 

Individual Researcher Prize
Winner: Matthew Grubb (King's College London)

 

Team Prize
Winner: Experimental Design Assistant development team (NC3Rs)

Highly Commended: Cam-CAN (University of Cambridge)

Most preclinical scientists would acknowledge that research practice needs to improve to address issues of reproducibility, yet the uptake to date of Registered Reports has remained very low. In this blog, CAB member Anthony Isles looks at potential reasons why there is this current disconnect, and what can be done to try and change the perception of Registered Reports in the preclinical research community. 

The BNA has launched a survey to consult the neuroscience community on the UK's research assessment system. This will help inform our response to the broader review being led by the four UK higher education bodies, the Future Research Assessment Programme, which will recommend changes to the current system.

Changing the Research Excellence Framework reflects the commitment in our Credibility in Neuroscience manifesto to ensure that the influences which drive research also drive the most credible research. Respondents to our consultation should complete this by 15 April. 

The Future Research Assessment Programme (FRAP), established by the four UK funding bodies (Research England, Scottish Funding Council, Higher Education Funding Council for Wales, Department for the Economy, Northern Ireland), is seeking views on the design of the UK’s future research assessment system.

Responses to this consultation will be one of the key inputs that shape FRAP's review of the Research Excellence Framework. They will be considered alongside a range of internal and commissioned evaluations, feedback on the current exercise (gathered separately from institutions and individuals), and advice from the programme’s international advisory group. The deadline for responses is 6 May. 

In a survey of nearly 300 researchers, a new preprint looked at the current level of adoption of open science practices in neuroimaging, suggesting that this is growing but also has a number of barriers impeding progress. 

Although half of the participants were experienced with preregistration, the willingness to preregister studies in the future was modest. The majority of participants had experience with the sharing of primary neuroimaging data. This was also different depending on sector and location, withEU-based researchers having a higher need for data governance, while researchers in the health sector reporting a higher need for data governance and a more unsupportive environment.

In September, the BNA responded to a Parliamentary inquiry on reproducibility in research, highlighting that a wide variety of stakeholders need to take action to strengthen the overall system, support research careers, and boost credibility. 

The Committee has now published all its written submissions - read more about the key points of our response here. 

The BNA journal, Brain and Neuroscience Advances, has joined a newly established initiative aimed at supporting how Registered Reports are reviewed and published , and giving more flexibility and control to authors when publishing their work.

The Peer Community In (PCI) initiative is a non-profit, non-commercial platform that publishes the peer reviews of preprints. Its new PCI Registered Reports community is dedicated to receiving, reviewing, and recommending RRs across the full spectrum of STEM, medicine, the social sciences and humanities. Joining the initiative boosts the Journal's existing Registered Reports offering. 

Read the full article.

The Association of Research Managers and Administrators has heard at their 2020 conference that universities and researchers must publish their research openly to help improve the research culture in academia. Panel experts have agreed that this practice of open science will help to promote a more positive environment in academic workplaces.

The traditional metrics currently used to assess research performance have led to an increased pressure on researchers as well as a lack of progress in research. It has been suggested that if universities can shift their strategies to more open science practices then there will be an overall improvement in research progression and research culture.

Read the full article.

PLOS Biology has published an article looking into the pre-registration of animal study protocols. Although this practice has been widely supported, uptake has remained low and must be more widely incorporated across researchers and policymakers. 

The authors have suggested that implementation of pre-registration could be encouraged by funders and journals, suggesting pre-registration should be mandatory and a prerequisite for publication. Currently, however, the main burden is on researchers through administration. The authors hope that easing of this burden through online guidance and workshops will help increase awareness of this initiative. 

Read PLOS Biology's article.

The UK Reproducibility Network (UKRN) has received £8.5 million in funding to promote open research. This funding, which includes £4.5 million from the Research England Development (RED) Fund while cover the coming five years. This funding highlights how Research England are committed to keeping the UK at the forefront of open research practices. 

The primary aim of this funding is to drive open research practices across the UK, with institutions able to deliver training on this sort of research practice and evaluate its impact. The benefits of open research are recognised by the UK Government R&D Roadmap as contributing to the improvement of research culture.

Read UKRN's article.

PLOS have posted a report showing how researchers value both impact and credibility when reviewing research. The study, which has not yet been peer reviewed, interviewed 52 cell biologists asking what they look for when conducting their own research and when involved in committees in the hiring or grant approval process. The data confirmed that credibility was the dominant factor considered during researcher's own work, and although less represented still strongly considered in the committee process. In addition to credibility and impact, researchers also focus on how reproducible research is.

Although these are preliminary findings, with follow up work expected in the future, these insights highlight the importance of credibility within research and can be utilised to improve research assessment guidelines.

Read the PLOS Blog's article.

The Department for Business, Energy & Industrial Strategy is calling for evidence and seeking views on ways to substantially reduce research bureaucracy. The goal of this review is “to advise on a substantial reduction in unnecessary research bureaucracy in government and the wider sector, supporting our researchers to focus on research and related activities which contribute to a healthy research base”. Outputs from this review will support the government's Research and Development Roadmap.

The review will focus on Higher Education Institutions and research organisations. While the government acknowledge the importance of research undertaken by businesses, this is outside the scope of this review. The review will build on initiatives already being undertaken by funding bodies and research organisations, such as UK Research and Innovation’s ‘Reforming Our Business’ programme, and informing the forthcoming review of the Research Excellence Framework.

Submissions are invited by 01 October. Read the full consultation.   

The House of Commons Science and Technology Committee has announced it will be looking at reproducibility and research integrity as the subject of a new inquiry. In its terms of reference, the Committee considers that "the specific issue of reproducible research has been overlooked" within the overall theme of research integrity. 

The Committee has issued a call for evidence on a number of areas including: the scale of the reproducibility crisis and factors in academia that have led to this; how funders/researchers/publishers/government can respond; examples of positive change in this area; and how might reproducibility be tackled by a new national committee on research integrity. 

Written submissions are invited by 30 September. Read the terms of reference.   

The UK Government has published a new R&D People and Culture Strategy, issuing a call to action inviting the sector to work with them to drive lasting change on people, culture and talent.

The Strategy includes commitments to develop a "New Deal for post-graduate research students", to provide support for flexible, cross-sector training programmes to encourage more movement & collaboration between academia, industry and the third sector, and to ask UKRI to undertake a review of how they use expert peer review.  

Part of the strategy also includes encouraging take up of the narrative researcher CV based on the Royal Society's Resume for Research to recognise a broader range of research contributions. A number of funders including UKRI and NIHR have alongside the strategy committed to explore a joint approach on a narrative CV. 

Read the new strategy. 

A new paper published in Nature Human Behaviour has found that reviewers rate registered reports as more rigorous, and their methods as higher in quality, than similar papers published in the standard publishing format, as well as these being considered by reviewers as being as creative and novel as standard papers.

353 reviewers from psychology faculties in the United States and Europe were recruited for the study, and were allocated published registered reports. Each reviewer evaluated a report and a matched, standard paper from the same journal or authors. The research team, led by Brian Nosek,  plans to build on this small study with a large randomised controlled trial of the effects of registered reports.

Read the Science article.  

A new study published in Science Advances has suggested that scientific research findings that are probably wrong gain far more attention than robust results, with the authors suspecting that the bar for publication may be lower for papers with "grabbier" conclusions.

The researchers from University of California in San Diego analysed how often studies in the three major replication projects were cited in later research papers. Studies that failed replication accrued, on average, 153 more citations in the period examined than those whose results held up. Only 12% of the citations acknowledged that replication projects had failed to confirm the relevant findings.

Read the Guardian's article.  

UKRI has today announced the launch of the Future Research Assessment Programme to explore possible approaches to the assessment of UK higher education research performance. 

Led by the four UK higher education funding bodies, the programme seeks to understand what a healthy, thriving research system looks like, will evaluate REF2021, and seeks to identify models that can encourage and strengthen the emphasis on delivering excellent research and impact, and support a positive research culture.

Advised by an International Advisory Group headed by future International Science Council President Sir Peter Gluckman, the programme expects to conclude its work by the end of 2022.   

Read more on the Future Research Assessment Programme.  

The Russell Group, representing 24 leading research-intensive UK universities, has published a new report that examines the drivers and incentives that shape UK academic research culture.

Realising Our Potential is based on conversations and interviews with almost 100 representatives from universities, funders and publishers - and is published alongside a toolkit of best practice from its members. 

Read more on the Russell Group website or the Wonkhe blog on the report. 

This editorial from Nature looks back at progress made by the Journal since 2013 around increasing transparency and reproducibility, highlighting how it has introduced reporting checklists, which are provided to peer reviewers and published with each life-sciences paper. 

This also draws attention to the MDAR (Materials Design Analysis Reporting) Framework, which has just been published. MDAR aims to encourage more-detailed disclosures in four areas of life-sciences manuscripts: materials (such as reagents, laboratory animals and model organisms); data; analysis (including code and statistics); and reporting (adhering to discipline-specific guidelines). 

DOI: 10.1038/d41586-021-01167-9

Read more on MDAR.

The BNA carried out a survey in autumn 2020 where views on Journal Impact Factor (JIF) and different models for peer review at its journal, Brain and Neuroscience Advances, were put to respondents. 

In this Editorial, the results of the survey are highlighted that JIF remains important to neuroscientists, particularly ECRs, and a clear majority were not supportive of tactics that can be used to boost JIF by a journal. On peer review, there was support for the BNA Journal to introduce alternatives to single blind review. 

DOI: 10.1177/23982128211006574 

PLOS ONE has placed an emphasis on soundness, as opposed to novelty, in published research. However, a study of peer review reports from PLOS ONE, reviewed by the LSE blog, highlights some resitance within peer review culture impacting PLOS ONE’s efforts to change academic culture more broadly.

The journal has sought to value elements such as reproducibility rather than assess novelty, yet the study found reviewers for PLOS ONE frequently comment on the novelty of the papers they review, rather than remarking upon reproducibility. 

We are delighted to congratulate the winners of the inaugural BNA Credibility Prizes, selected by our Credibility Advisory Board: 

Winner: Marta Topor (University of Surrey)  

Highly Commended: Alberto Lazari (University of Oxford), Alexandra Lautarescu (King's College London)

 

Individual Researcher Credibility Prize 2021 

Winner: Amy Orben (University of Cambridge)  

Highly Commended: Jade Pickering (University of Southampton), Saloni Krishnan (Royal Holloway, UoL)

 

Winner: CAMARADES (University of Edinburgh)

Highly Commended: Wellcome Centre for Integrative Neuroimaging (University of Oxford) 

Find out more on the prize winners.

Cohen Veterans Bioscience and the European College of Neuropsychopharmacology have opened a call for submissions for the 2021 Best Negative Data Prize in Clinical Neuroscience. The prize recognises the researcher or research group whose publication in clinical neuroscience best exemplifies clinical data where the results do not confirm the expected outcomes or original hypotheses or results that challenge a long-standing clinical precedent.

Submissions are open until 30 April 2021.  

The BNA is pleased to announce our endorsement of the Hong Kong Principles for assessing researchers.  

If you're thinking of presenting a prereg poster at an upcoming conference, you can now add a new badge to help highlight this poster type, and help encourage others to use these in the future.  

The winner of The Neuro's international Open Science in Action prize has been awarded to Daniel Aharoni, an assistant professor from The University of California, Los Angeles for developing an open-access resource in neuroscience. Aharoni developed a miniaturized microscope to record the activity of hundreds of neurons simultaneously in freely moving animals

The Neuro Open Science in Action Prizes are part of an initiative in its second year from The Neuro’s Tanenbaum Open Science Institute at McGill University in Canada. The prizes recognize projects, services, tools, and platforms that unlock the power of open science in neuroscience to advance research, innovation, and its application for the benefit of health and society.

The very first Registered Report has this month been published in the BNA journal, Brain and Neuroscience Advances, marking another milestone in our commitment to supporting Credibility in Neuroscience. 

The Registered Report tested the antagonistic pleiotropy hypothesis of Apolipoprotein E (APOE). Interestingly, it did not find evidence to support this hypothesis, indicating that it may be persisting due to publication bias in the field. 

DOI: doi.org/10.1038/d41586-020-02275-8

Building on the ARRIVE guidelines developed by NC3Rs in 2010, the ARRIVE 2.0 guidelines splits recommended practices in two: the “ARRIVE Essential 10” providing the basic minimum information to include in a manuscript for readers and reviewers to assess the reliability of the findings, and acomplementary “Recommended Set” that provides context to the study. 

Having introduced and evaluated the effectiveness of pre-registration posters at the BNA's 2019 Festival of  Neursocience, this article in Nature Human Behaviour, sets out the preliminary evidence, collected at the Festival, showing how preregistration posters are valuable in providing feedback, promoting open science, and supporting early career researchers (ECRs).  The publication should encourage uptake of initiatives such as this, which aim to support reproducible, credible and open neuroscience, by the research community.

“This is a massive achievement for the journal, strengthening our position within the scientific community, as well as supporting and promoting the BNA’s commitment to credibility in neuroscience." - Professor Anthony Isles, BNA trustee for communications and publications.

DOI: https://doi.org/10.1186/s41073-020-0091-5

DOI: https://doi.org/10.1186/s13041-020-0552-2

The Lancet reports on the BNA's Credibility in Neuroscience initiative
DOI: https://doi.org/10.1016/S1474-4422(19)30481-8

"Only if changes occur across many institutions will the impacts permeate scientific culture." 

Marcus Munafo, co-founder of the UK Reproducibility Iniatitive (UKRI), discusses flaws in our institutions’ cultures and practices that allow questionable research practices and undesirable behaviours arise and persist.  Only if institutions pull together can change be achieved. 

DOI: 10.1038/d41586-019-03750-7

Head of REF policy at Research England, Helena Mills, said that although the organisation had repeated “until we’re blue in the face” that journal impact factors were not used in assessments, people “can still turn around and say, ‘You do though, don’t you?’”