Improving research with registered reports

Cancer

We’re launching a new scheme, in collaboration with the University of Bristol and a consortium of journals, to streamline the way researchers can publish their research, irrespective of the findings. Here we chat to Professor Marcus Munafò – a member of one of our funding committees – to find out what a Registered Report is, how you can get involved, and why this relatively small change in the way research is done could have a real impact on reproducibility issues and even patient outcomes.

So, first up – what exactly are Registered Reports?

Registered Reports are an article type offered by an increasing number of journals. When published, these articles look no different to a conventional article. What is different is the process they’ve been through.

Registered Reports are reviewed in two stages. At stage 1, the study protocol is reviewed – roughly, the introduction and methods of what will eventually become the full article – before any data have been collected. The focus is on whether the research question is important and the methodology robust. If the article passes stage 1 review it is given ‘in-principle’ acceptance, which means that it will be published once the data have been collected regardless of the eventual outcome.

The stage 2 review is a light touch check. In principle, this approach ensures that studies are designed to be informative regardless of the nature of the eventual results and removes incentives to focus on positive results only.

Will this way of reporting their work involve more work for researchers and what benefits could they see by participating in the pilot?

It’s more a matter of the work being in a different place, with much of the writing and the peer review happening before data collection rather than afterwards. Since the focus is on reviewing the study before data collection, reviewers can recommend changes at the point where these changes can still be incorporated.

There are therefore reasons to believe that the quality of Registered Reports may be higher than conventional articles. In addition, once ‘in-principle’ acceptance is offered the authors can be confident that their work will be published, and rapidly, once data collection is complete. It’s important to note that deviations from the stage 1 submission are permissible, but these need to be justified and described clearly as such. Sometimes this requires additional review. But overall, the hope is that Registered Reports will support higher quality work and more rapid publication.

Registered Reports have been offered from a number of journals for a while, why is involving the funders an important step?

It’s great that journals offer Registered Reports, but it’s important that there are incentives for researchers to try this format. By joining up journals and a funder to create a single, coordinated process, the hope is that applicants for funding will be encouraged to try Registered Reports.

The rationale is that the study protocol required for stage 1 submission is similar to a grant proposal, and the logical next step after funding has been awarded is to develop the study protocol. It’s a small step from there to submitting the protocol for consideration as a Registered Report. There is also scope for greater efficiency – for example, if some of those who reviewed the grant also review the stage 1 submission, this will be a relatively small amount of work compared with seeing the submission for the first time.

Do you think every area of cancer research is equally applicable to this way of working?

Registered Reports work very well for some study designs – particularly those where new data are being collected to test a focal hypothesis. But they can be surprisingly flexible – the basic approach has been used for multi-experiment studies, observational studies, secondary data analyses, and qualitative research. It’s really for researchers themselves to decide whether the format works for them, but the format isn’t intended to be prescriptive. The principle of the methodology being reviewed before data collection is the important thing, as well as the publication decision not being dependent on the nature of the results, but rather on the importance of the question and the robustness of the methodology. This can certainly be applied to different types of research.

This way of doing ‘open science’ could really help with aspects of the ‘reproducibility crisis’ – do you think it’ll be enough?

I don’t really like the “crisis” narrative. What we’re seeing across the sector are lots of innovative approaches to how we conduct our research being developed and piloted. The focus, for me, is thinking about how we can continuously reflect on, and improve, our working practices to ensure our research is of the highest quality possible. This focus on improving quality should, hopefully, also mean that the knowledge we generate translates into clinical or societal benefit more rapidly. No single approach – including Registered Reports – will be a silver bullet though.

In general, I think transparency in research is valuable for a number of reasons, and Registered Reports are part of this. But it’s also important to remember that we will have to evidence these new approaches to see whether they do in fact work as intended, and check that they don’t have any unintended consequences. Hence the need for meta-research – or research on research.

Could we even go so far as to suggest this could help with speeding up the patent benefit of research?

There are good reasons to believe that focusing on improving the quality of research outputs should also speed up the translation of that knowledge into patient benefit. Science self-corrects, eventually, but it will do so more efficiently (and have less need to!) if we generate more robust knowledge in the first place. We’ll always need to balance the need to generate robust evidence with the need to take risk and pursue new directions, but in my view that balance isn’t quite right at the moment.

Ultimately, this is an empirical question we can answer by collecting and evaluating evidence. With the variety of innovative approaches to research and publication that we’re seeing emerging – including Registered Reports – we should eventually be able to determine whether these new ways of working do in fact produce more robust findings, and in turn whether this speed up translation into patient benefit.

Want to find out more about Registered Reports? Join our webinar on 19 May

You can find out more about Registered Reports here and our pilot scheme here

Marcus is Professor of Biological Psychology and MRC Investigator at the University of Bristol. He has a long standing interest in the factors that influence research quality, and in 2019 co-founded the UK Reproducibility Network, and along with colleagues at the University of Bristol is a key part of the Registered Reports consortium.

He is currently Editor-in-Chief of Nicotine and Tobacco Research, and a member of the Cancer Research UK Prevention and Population Research Committee.

More on this topic

Tags

There are no tags

Leave a Reply

Your email address will not be published. Required fields are marked *