Despite an abundance of journal outlets, a rising number of published articles, and regular peer review congresses, at least one kind of peer review research appears to be underrepresented. Vying for attention and feeling overlooked, interventional studies aimed at improving peer review and scientific reporting often remain unpublished. Other kinds of research into peer review methods fare poorly, too. For example, studies exploring reporting guidelines, synthesis of evidence, and publication bias also struggle to become part of the biomedical literature.
These sobering findings were reported in a peer review study published March 12 in the Journal of the American Medical Association. In an article entitled “Study Design, Publication Outcome, and Funding of Research Presented at International Congresses on Peer Review and Biomedical Publication,” peer review researchers from Croatia and Switzerland scrutinized the content of presentations at their discipline’s most prominent meetings. In addition, they investigated how frequently studies presented at the peer review congresses (PRCs) attracted funding or appeared in professional journals. Their analysis took in 614 abstracts that appeared at seven PRCs held from 1989 to 2013.
In their article, Mario Malicki, M.D., of the University of Split School of Medicine, Split, Croatia, and colleagues reported that 39% of studies presented at PRCs have not been fully published. “In our cohort, we were unable to determine whether the underreporting was selective (e.g., favoring positive results) and were not able to determine its causes.” According to the researchers, funding was reported in 106 (36%) published articles that had been presented as abstracts at the 1989–2009 PRCs and in 45 abstracts (41%) presented at the 2013 PRC, most commonly from public or charity sources.
With respect to publishing outlets, the researchers noted that no decrease the publishing output of journals has occurred since JAMA ceased its PRC theme issues in 2005. Out of 294 abstract presentations from the first six PRCs that reached publication, 114 articles (38%) appeared in JAMA, 21 (7%) in BMJ, 12 (4%) in Annals of Emergency Medicine, and 8 (3%) each in the Journal of Clinical Epidemiology and in PLoS ONE. The median time to publication was 14 months, when excluding 110 articles in JAMA theme issues.
“Peer review and other editorial procedures have the potential to significantly influence the knowledge base of health care,” the authors wrote. “Despite their critical role in biomedical publishing, methods of peer review are still underresearched and lack dedicated funding. Systematic and competitive funding schemes are needed to build and sustain excellence, innovation, and methodological rigor in peer review research.”
In a JAMA editorial inspired, in part, by the present study, Drummond Rennie, M.D., of the University of California, San Francisco, and Annette Flanagin, R.N., of JAMA, Chicago, observed that “articles on how to improve research, of which publication is an integral part, are important reminders that no matter how much research on peer review and publication has been presented at the Peer Review Congresses and elsewhere, these studies are but part of a widespread movement to improve the scientific literature. As the reports in this issue of JAMA indicate, discovering the extent of the problems and testing methods to correct them will require a massive and prolonged effort on the part of researchers, funders, institutions, and journal editors.”
