As many work to make “open science” standard practice, an often overlooked area is whether the scientific research and publishing process is open to all stakeholders. In health and medical research, that means including patients and the public in the research that will significantly impact their lives. Bringing together academics and patients is the mission of the BMC journal, Research Involvement and Engagement,1 as exemplified by their founding co-Editors-in-Chief, Sophie Staniszewska, a Professor of Health Research at the University of Warwick, and Richard Stephens, a patient advocate. Sophie and Richard recently spoke with Science Editor about the necessity of accessible, understandable research, the importance of community engagement, and the need to democratize research through public involvement.
Science Editor: What led to the creation of this journal?
Sophie Staniszewska: The origins of the journal were back in 2013–2014, where I was working with a group from the National Institute for Health Research (NIHR) INVOLVE and thinking a lot about evidence and knowledge and learning as important adjacent concepts. We realized that people were often undertaking projects and wanting to publish their involvement work, but journals didn’t always accept papers about involvement. There was a real gap in the market for researchers to publish this sort of work. That’s important because we want to build an evidence base for practice, and you need to be able to publish so people can refer to and cite work to use it.
Together with a group from INVOLVE, we submitted a proposal to BMC (now BMC Springer Nature) to launch a journal that addressed this gap. We launched in 2015, and as part of that process, we agreed that we needed a co-Editor-in-Chief to represent patients. I rang Richard because he was top of my list. Luckily for me, after some thinking, he said yes.
Richard Stephens: I have a slightly different genesis to the whole journal. Sophie is right in how it happened, but the other side of the coin is the conversations that she and I and other people on the patient and public involvement circuit in the UK were having around NIHR meetings in Southampton. Sophie did all the groundwork and all the hard work, but nevertheless, there was this general feeling from patient advocates that we wanted somewhere to publish our stuff. We didn’t want it only to be in newsletters or blogs. We wanted the credibility that comes with a proper peer-reviewed academic journal, but we also wanted to be part of the peer review process. We hadn’t thought about editing it—I certainly hadn’t—but that was a logical outcome: a co-produced academic journal that would have the kudos for our work.
Science Editor: How do you define the role of the patients in contrast with the academics?
Richard: It was probably easier then than it is now. Then [at the start of the journal] there was a kind of simple definition because all of the academics were working in clinical or health services research or for academic institutions, and we patients, by and large, weren’t; we might be working with them, but we weren’t employed. We did not have contracts with universities. We were not employed in the health service. That then was the rough division. Of course, there are individuals who do cross those boundaries: a doctor can also be a patient and then they come to it with two hats on. But it was really the people who were involved as patient or public representatives in existing research projects. That was fairly easy to define. Are you involved in a research project? “Yes.” What is your role? “I’m the patient representative.” As opposed to: “I’m the statistician or I’m the chief investigator.” Much of it was self-defining.
Sophie: The definitions are often hard to exactly pin down. One person can have more than one identity. If you’re a patient, you might be seen as a service user of mental health services, you might be a caregiver, or you might also be a community representative who is very active in some areas. In research, we often don’t spend enough time exploring that identity and what people bring until it manifests itself in their comments. I feel very comfortable with this fluid definition because it changes over time; as people pick up more experience and they do more things, they see themselves in different ways. As Richard says, sometimes we find that academics are also patients. There are some professors of mental health research who are also service users, and when they present and when they write, they bring those two elements together, which is very powerful. I think in a way, we need to enable people to find their own identity—if you like, if they probably know already—but to value that and to try and understand it and to appreciate what impact that has on what they contribute to research, because it’s going to be different for different people, with different experiences.
Richard: We do have reviewers who appear on our lists both as academic reviewers and as patient public reviewers. There’s a friend of mine who has published several papers, including in our journal, as a patient researcher. And as he asks, what is he now? Is he still a patient? Well, yes, he’s still having treatment, but he’s published more papers than some professional researchers.
Sophie: In the UK, we’ve also had a user-led research movement that has been very important in developing the involvement movement, and it was often mental health service users who were also academics leading pieces of research, but very much through that lens. That did give it a very different critique and a different approach. In a way, it’s almost like an ecosystem where the more diversity we have, the stronger it is because there’s the full range of perspectives that we’re accessing. We try not to be too worried about absolutely specific definitions because we recognize they change and they’re fluid.
Richard: It’s also becoming even more blurred now because there is more patient and public involvement in basic science, in laboratory-generated research, and in genomics. That has stretched it because most of us, even 6 or 7 years ago, would never have thought of ourselves as scientists. Even if we thought of ourselves as clinical or qualitative researchers, scientists deal with science, but more and more of us patients are now doing that too.
Sophie: That takes us to a very interesting question of what is science, and what is knowledge, and what is evidence. These are contested as well. Science and its concepts can still be seen as social constructs: someone has decided a concept is important or developed it at some point and there is a dialogue to be had around who made those decisions and who decided it should look like that. That means in areas where you might not expect public involvement, it can still happen given the right context. For example, we’ve been working on a study looking at how the public are involved in mathematical and economic modeling. That’s very common now with COVID, of course; however, when we started, it was a very hidden area but one where we were keen to explore the potential of involvement.
As you unravel it with your public contributors and you have conversations about it, you realize it is a social construct with lots of decisions about what variables go into a model, how they’re construed, how valid they are, what they represent, and how they’re combined in ways that create interpretations that someone else then puts into policy. From that, we’ve developed a framework to guide other modelers. Our approach to PPI in more complex areas is always to go off and explore what the possibilities are and to not shut it down too early and think that you can’t include public involvement in a particular area. I would want it to apply to all journals; for me, public involvement is in many respects a paradigm shift: it’s about democratizing research. It’s about making research available for everyone and understandable for everyone and an opportunity to participate in that research. It’s a bit of a cliche, but together we are better because we bring that broad well-rounded perspective to the topic of interest. If it is only academic researchers looking at it, they may miss some really important factors that will impact people’s lives further down the road.
Richard: Sophie’s right: this is about democratizing research, not only research studies, but the whole research environment, especially as we fund a lot of it—taxation or donation, it’s our money. Involvement now includes getting involved with funding and priority decisions, whether it’s for a research team or a national strategy. It involves sitting on things like data and safety monitoring committees for interventional studies. That’s an area which has had very little involvement and certainly next to nothing published, but that is an open door and people are beginning to go through it. And it involves writing papers!
Sophie: The reason we’re doing all this is because it’s about the quality of the research. It’s about asking the right questions. It’s about measuring the right things in the right ways. It’s about interpreting the results appropriately and understanding their full potential impact. It’s about enhancing the conventional concept of validity in research with the idea of community validity, but it’s also about choosing studies or topics that have relevance and potential impact on people’s lives. Us academics could study all sorts of things, but whether they make a difference to someone’s life is questionable. As Richard said, when you have a public paying for that research, there is an ethical and moral accountability to deliver research that makes a difference to people’s lives.
The quality argument is that you can feel much more confident that you have undertaken your project in a way where you’ve considered all the relevant concepts, and that you’ve thought about the sorts of things your public contributors are thinking about. At the end of it, the study is a better study for it. It’s higher quality in conventional research terms, but from the public perspective, it’s more likely to go off and create some useful impact. Certainly, now we see that a lot of interest in patients wanting to implement the study results; to take the outcomes of a study, go to their local hospital and say, what about this? Why aren’t you doing this? Or please do this for us. Then there’s even more focus and interest for the research to be relevant.
If you go to your doctor and you want treatment, you want to make sure your discussion with your doctor about which treatment would be best is based on outcomes of relevance to you. Otherwise, your discussion could be missing the mark and giving you something that’s not going to work—making sure things like outcomes are the right patient-important outcomes, measured in the right way and not just psychometrically driven instruments that work well psychometrically, but don’t measure anything of importance. All these things come together and it’s quite a complex picture of different motivations and different reasons, but with that sense of trying to make research better and to create more patient benefit and better health and better outcomes.
Richard: That does lead to another growth area in terms of patient involvement, which is in influencing regulators about their decisions, particularly around quality-of-life measures that they use, for example, to judge whether or not a drug is worth funding. Also clinical guidelines in the UK and in Europe, where big conglomerates like the European Society of Cardiology are producing guidelines for clinicians across Europe, and patients are getting involved in producing the next iteration of those guidelines. For us, we usually get involved because someone we care about or ourselves has had a health problem; it might’ve been resolved, but that’s usually, not always, but usually why we get involved. We have the phrase evidence-based medicine, and in the past, patients have been interested in the medicine and the researchers have produced the evidence, but now there’s much more crossover.
Sophie: To pick up on that, one of the concepts I was involved in developing is patient-based evidence. In a way, some of this movement is about reconfiguring what we think of as evidence. We’ve been working with colleagues in Health Technology Assessment (HTA) around this concept of patient-based evidence. We published in a special issue2 of the International Journal of Technology Assessment in Health Care this year looking at the patient and public involvement elements in HTA. We’re arguing that we need a fundamental rethinking around what evidence should look like and who creates it and how it’s cocreated as well. Again, those elements come into it because we want people to be creating studies that measure evidence of different sorts and making sure that it’s relevant to the question and to the people involved. We’re trying to push some of those boundaries as well.
Richard: It’s about finding the better balance, particularly in illnesses like cancer, which is admittedly my background, but so many studies still focus on progression-free survival, where they’re looking to prolong life often by months—not years, months. Increasingly, patients and patient families are saying, yeah, that’s all very well if you can prolong the survival, and of course you can measure that, but what about the quality of life? What instruments are you using to measure that, and were they designed 30 years ago when your survival rates were much, much less than they are now? There are real hard conversations being had because we don’t know what the answer is; what’s now happening is that we patients are in the room asking that question. Why do you have to have one primary endpoint? Why can’t you have two and patients can see the trade-off between, say, length of life and the quality of it?
Sophie: That also raises the point that we’re not talking only about patient and public involvement in the content of research; we’re talking about it in the context of methods and methodological development. The question of whether you are measuring quality of life in the right way for this group of patients is a bigger question. It’s a question almost about whether you have the right methods to develop your instrument in ways that will address that question.
Quality of life measurement has been dominated by methods that are very good, and they test and develop instruments in really helpful ways. What we haven’t seen in the same way is the embedding of public involvement at each stage of development of patient-reported outcomes. As a journal editor, I’m always looking out for papers that have done some methodological work that gives the rest of the community a sort of a leg up in terms of understanding the potential for something methodological. We’ve still got quite a long way to go as the funding for this sort of work is mostly nonexistent. It’s really difficult to get funding for public involvement in methods development. We have funding opportunities for public involvement in content of research and NIHR is hugely supportive of that, as are other UK and international funders. We should be linking public involvement and research more strongly through methods and methodological work as well. On my wish list for the next decade would be that we’ve really addressed that and moved forward. It will help in the same way as I hope that our framework for public involvement in mathematical and economic modeling means the next person to ask the question of how we involve the public in those discussions will have somewhere to start. They might not use all the bits of our frameworks; it’s not meant to be prescriptive, but they might take something from it that creates a conversation about a model and its appropriateness for a particular patient group that might change that outcome. As editors, we have a role to publish work, but also to look out for work that is pushing forward on thinking as well and creating dialogue and debate.
Richard: That in itself is a challenge because if something is around a methodological issue, that does limit the number of patient reviewers who can usefully review such a paper, because most of us are actually looking at outcomes. We do look at methods. It’s often about how do you deliver a clinical trial, not how do you construct a valid quality of life instrument, let alone how do you persuade other people to adopt and validate a quality-of-life instrument? This is one area where patient desires may persuade researchers to change the way they do things, but then we’ve got to get patients and patient groups to catch up with that issue and the current methodologies, even though we were the ones who asked for it in the first place. It’s a really interesting conundrum, and we do struggle. We need better conversations between academics and patients about methods to help us move forward.
Science Editor: This discussion about identity and involvement in the trials reminds me of the similar push around diversity and inclusion in clinical trials. New England Journal of Medicine recently had an editorial3 about ensuring that clinical trials include participants from the populations affected by the disease or treatment they are studying, because different populations are affected differently. I see how this ties into a broader sense of thinking about who is being studied and how they will be affected by the research being produced.
Sophie: Funders have a big focus on trying to enhance the diversity of participants in studies, but also within the public involvement arena. That’s taking us into areas like community engagement as well, because to create those relationships with communities and create the diversity of involvement in research, you’re then looking at very different ways of working in terms of not just one project where you’re inviting public contributors, but also you’re looking at longer term high-quality relationship development that is also about reciprocity and about addressing issues of concern to those communities. There’s a bit of a transition, I think, starting in research and the way funders think about this. Probably in the future, they’ll have to change some of the expected ways of working, because those long-term relationships aren’t supported by single projects.
It’s easier in centers when you’ve got a five-year-old funded center, then you can do more of that work; even then, it’s high risk because at the end of the five years, you’re effectively saying, that’s it folks: We’re finished, but the community may want to continue. It’s challenging academics to think about how that will work, but also the communities will probably need more of a voice in this. At the moment, there is an effort to go out and connect with people, but less of a strategic focus on how we do that, which is something we will need to develop.
Science Editor: We’d like to switch now and discuss how the peer-review process works with patients and public reviewers. How have you found that they deal with the specialized language and particularities of a scientific article that can be unfamiliar to some patients?
Richard: We’ve had very few comments from patient reviewers saying a paper itself is too complicated to understand. I think there are two reasons. One is that by and large, all of our reviewers are experienced in working in research. That’s how and why we’ve recruited them—because they’ve already been on papers, or they are from the European patient academy, or they are from lists of patients like the National Cancer Research Institute consumer forum in the UK or in other countries. These are by-and-large experienced patient advocates already working in research. The second reason is that we insist every paper comes with a plain language summary. We have a rule that if Sophie or I can’t grasp the plain language summary, the paper itself goes back to the authors: “Rewrite the summary. Oh, and by the way, while you’re rewriting the summary, you might want to rethink aspects of the paper.”
We want our papers to be read by patients and the public. Many of them are readers, not reviewers, and the readers will struggle with 18 pages of academic language. I do, myself, and I’ve been doing this for 20 years and have a university degree. This is not easy stuff. So, the plain language summary makes it accessible to reviewers and readers alike. Often, some of our reviewers will tell us that the plain language summary doesn’t actually match the paper. One of the good developments from that is increasing numbers of researchers, whilst they’re still writing academic papers, and their plain language summaries and their abstracts are now identical. If abstracts are written in plain language, and they are shared at conferences and other events where the public have access or where abstracts are published online, so much the better. We also live in an atmosphere where in Europe, for example, every clinical trialist is now required to register their trial on a publicly accessible website, and they have to have a plain language summary with it. All of those things make it more accessible for reviewers, but our reviewers are experienced and know what they’re doing, but we all find academic papers quite challenging.
Sophie: The other thing is we do encourage authors to write in ways that are accessible. Given a choice of a long sentence that no one can understand versus an easy to understand one, we’d always encourage them to think about using easily accessible language and defining terms if they have to use a specific research term, try and explain that or define it. I think our reviewers are pretty good at picking up when that’s not happening, and it’s difficult to understand. That’s fed back to the authors, and they can adjust it so you’re not losing the essence of what a study is about; you’re just making it more accessible to more people so they can take those findings and use it. I think also being open access means anyone can access our papers, because I know there is a huge frustration in the patient community because papers are behind paywalls, and it might be papers those patients have been involved in, or it might be their idea. That’s really difficult then to hear that it’s behind a paywall, and you can’t access it. I think a big plus for us is that anyone can read our papers, hopefully anyone can understand them, and our system is set up to support that vision of trying to create an understandable paper for everybody.
Richard: Our readers can also, of course, read the peer reviews online, so if there is part of the paper that is hard to understand, and that’s been picked up by reviewers, they can actually see that; even if the authors have not changed the paper, they can see the authors’ response to it.
Sophie: I think you have to recognize, in a way, that the nature of what we’re producing, by its very definition, demands that it’s accessible because it’s about developing our knowledge and evidence about public involvement. It would be a slight sort of our own goal if it wasn’t understandable. Also from a publisher perspective, you haven’t just got a small group of people interested in your journal. You’ve got everyone in the world interested in it, potentially, which is I think very attractive. The other element here is the interdisciplinarity so that any academic can read our journal and understand it.
Richard: It would be very ironic given that our journal is about involving and engaging the public and patients if an academic gave us a paper that was impenetrable, but so far nobody has. Or rather, we haven’t published one!
Sophie: I think that’s a testament to the community that people do get that, and they are respectful of it. They’re often working with public contributors who may be part of the writing, so the impenetrable language is slowly removed from a paper as part of the presubmission writing process. A lot of the papers come from funded research where the funders support this more plain way of working. The researchers have had to think in this way from the beginning, and they’ve had to write a plain English summary of their intent. It’s embedded at all levels, certainly in the UK context and a lot internationally now. We’re supporting a movement; we’re not creating a new one. We are part of a bigger picture that is, for me, a paradigm change in the nature of academia, but one that is a positive one that takes us beyond the small groupings of specialty we’ve had in the past, but actually creates a universal community of academics and patients and public working together.
Science Editor: How are patient and public reviewers invited to review manuscripts, and how can they express their interest in reviewing?
Richard: The mechanics of it are the same as the way we invite academics. There is absolutely no difference. It’s a computer-generated email. Some of the patient reviewers struggle with the computer system, perhaps some of the academics too. If they want, they can sign up on the website, and many of them have, so it isn’t really an “invitation”—they have volunteered. Of course, first they’ve got to learn about the journal, and the problem we have with the journal is getting across the concept of it. It is an academic, peer-reviewed journal. Every single paper we publish is reviewed by at least 2 academics and at least 1 patient (2, if we can get them). It’s the same as any other journal. It’s academically peer reviewed, but the papers are accessible to the public, partly because of the plain English summary, but also the general structure of the papers. It’s not just the paper being open access; it’s the reviews being open access too. Getting that across to people is difficult.
Science Editor: Looking at the published reviewer reports, the academic and patient reviewers are not identified as such to authors and readers, correct?
Richard: No, we wouldn’t want them to be. The reviewer type is identified in the invitation letter that goes to them, but that’s partly because we do have people who are both so we’re trying to tell them which hat we want them to wear for this review.
It’s interesting because I’ve made some general comments about patient and academic reviewers, but as you can see on our website, you will find academics who have corrected the spelling of every single word in a manuscript and said little more. And then you’ll find patients who have written a 400-word review, which in effect says this is a really interesting paper, but you have missed so many opportunities. You cannot always tell which would be which.
Science Editor: Have you seen a change in the approach of funders, industry, and researchers to the inclusion of patients and the public in research since the journal started?
Richard: I think, yes. One or two groups have started to look at things like that, such as initiating clinical studies with pharmaceutical companies. This was patients going to a commercial company saying these are our concerns and what can you do to help resolve those concerns? That’s a brilliant model. As that happens more and more often, I hope we will get more and more papers. I think it’s natural the more we do things like biobanking and genomic research, patients who donate those samples or genomes are asking the research community, “What are you doing with them?”
Sophie: Same with health data. I think there’s much more of a movement towards active forms of involvement; not just building trust in health data, but actually working together to create a data set. I think in the last decade, there has been change, there’s been much more embedding and much more acceptance. Year-on-year the number of papers coming to us is increasing, and we see that for other journals as well who are publishing public involvement papers. I think there is an increasing movement that’s gathering pace and looking for the next challenge of trying to enhance the diversity of who works with us.
It’s an exciting time. As Richard said earlier, going online has created all sorts of opportunities for people. We’re working with public contributors who represent different communities, who we’ve recruited because of their community voice. They’re not necessarily NIHR experts with lots of expertise. They bring a different voice and they’re creating a greater diversity in the types of contributions people are making, which is really exciting.
Science Editor: That leads nicely to our final question: if there is a journal out there that is not typically involving patients right now, what recommendations would you have for those editors about how they can do that and what they should be looking for?
Sophie: It depends on the subject area. They could certainly come and talk to us about it. I guess doing things like establishing a patient or a public panel might be one way forward in the way the BMJ has. I think they could look at their publication system and look for the opportunities where patients or the public could be involved. I guess pragmatically, it’s about seeing how it could work. I think they’d need to consider what their vision is of involvement. For us, it’s about coproduction. They might want to follow or adapt it. I think they need to have patients or the public advising them on that, because I think it’s really hard to do that without that sort of expert knowledge of how it would be received in the community.
I would also encourage them to think about writing something, which is about that position so that people understand what they’re thinking. We’ve had other journals approach us, and we’ve had discussions with them and explained how we work. I think they’ve gone away and thought “this bit could work or that bit could work.” It would depend on the journal, but I think the advantages to them would be significant. I think all the things we’ve talked about today: about opening the journal up to wider scrutiny, that sense of democratization of knowledge, and creating useful knowledge for patient benefits. I think there’s lots of different things they could consider.
Richard: From the other end, I would ask the journal editors, what’s your journal for? Are you interested in publishing and continuing to publish very successfully lots of academic articles, or would you actually like to help democratize research and explain your science to the masses? If so, you have a role here, but it’s not just you: what are you doing to encourage your authors; to say, next time you do a piece of research, how about involving patients and the public or citizens from the start? Is there an opportunity here to involve citizens, that is, the people who participate in health research and who benefit from it?
Not because you think you ought to, or because Sophie and I are saying it’s a good idea but think more constructively about whether involving citizens actually adds value to your research. Would it add relevance to the people you want to read and act on your research? Would it help sharpen some of the questions or would it bring a completely new angle that you haven’t thought of? You won’t know until you bring the patients in the public and the citizens into your work. Journal editors could help revolutionize the world and make a better planet for all of us. That’s what they should be doing.
References and Links
- https://researchinvolvement.biomedcentral.com/
- https://www.cambridge.org/core/journals/international-journal-of-technology-assessment-in-health-care/special-issue-patient-and-public-involvement-in-hta
- https://www.nejm.org/doi/full/10.1056/NEJMe2114651
Janaynne Carvalho do Amaral is with Federal University of Rio de Janeiro and Jonathan Schultz is Editor-in-Chief, Science Editor, and Director, Journal Operations, American Heart Association.
Opinions expressed are those of the author and do not necessarily reflect the opinions or policies of the Council of Science Editors or the Editorial Board of Science Editor.