Comparison of self-citation by peer reviewers in a journal with single-blind peer review versus a journal with open peer review

https://doi.org/10.1016/j.jpsychores.2015.08.004Get rights and content

Highlights

  • We assessed peer reviews of manuscripts published in two psychiatry journals.

  • We compared reviewer self-citations in a single-blind versus an open peer review journal.

  • There was no difference between journals in proportion of self-citations.

  • There was no difference in proportions of self-citations with a rationale.

Abstract

Objective

Some peer reviewers may inappropriately, or coercively request that authors include references to the reviewers' own work. The objective of this study was to evaluate whether, compared to reviews for a journal with single-blind peer review, reviews for a journal with open peer review included (1) fewer self-citations; (2) a lower proportion of self-citations without a rationale; and (3) a lower ratio of proportions of citations without a rationale in self-citations versus citations to others' work.

Methods

Peer reviews for published manuscripts submitted in 2012 to a single-blind peer review journal, the Journal of Psychosomatic Research, were previously evaluated (Thombs et al., 2015). These were compared to publically available peer reviews of manuscripts published in 2012 in an open review journal, BMC Psychiatry. Two investigators independently extracted data for both journals.

Results

There were no significant differences between journals in the proportion of all reviewer citations that were self-citations (Journal of Psychosomatic Research: 71/225, 32%; BMC Psychiatry: 90/315, 29%; p = .50), or in the proportion of self-citations without a rationale (Journal of Psychosomatic Research: 15/71, 21%; BMC Psychiatry: 12/90, 13%; p = .21). There was no significant difference between journals in the proportion of self-citations versus citations to others' work without a rationale (p = .31).

Conclusion

Blind and open peer review methodologies have distinct advantages and disadvantages. The present study found that, in reasonably similar journals that use single-blind and open review, there were no substantive differences in the pattern of peer reviewer self-citations.

Introduction

Peer review is used by biomedical and psychological journals to evaluate submitted manuscripts and attempt to improve the quality of those that will be published. Peer review has a central role in the scientific process, but has been criticized as subjective, inconsistent, and potentially abused [1], [2], [3]. In addition to these concerns, some peer reviewers may make inappropriate, or even coercive recommendations to authors to unnecessarily cite the peer reviewer's own work [4], [5], [6]. Coercive self-citation via the peer review process may inappropriately highlight the peer reviewer's work and is in conflict with the essential role of the peer reviewer, which is to provide expert evaluation of a manuscript's merit, along with recommendations for improvement [5]. It may also put authors in the difficult position of having to decide whether to comply with superfluous requests or argue against such requests, which may reduce the likelihood that their manuscript will be accepted for publication [5], [6]. The Committee on Publication Ethics (COPE) Ethical Guidelines for Peer Reviewers indicate that “peer reviewers should not suggest that authors include citations to the reviewer's (or their associates') work merely to increase the reviewer's (or their associates') citation count or to enhance the visibility of their or their associates' work; suggestions must be based on valid academic or technological reasons” [7].

We recently evaluated peer reviews of all manuscripts that were submitted to the Journal of Psychosomatic Research in 2012, in order to assess the degree to which peer reviewer self-citation occurs [6]. The Journal of Psychosomatic Research uses single-blind peer review [8], where the identity of peer reviewers is unknown to the authors of submitted manuscripts. Of the 616 peer reviews included in the study, 171 included at least one peer reviewer citation, with a total of 428 citations. We found that 29% of all citations recommended by peer reviewers were citations to the peer reviewer's own work. Self-citations were more than twice as common in reviews recommending revision or acceptance, where 33% of all citations were self-citations, compared to reviews recommending rejection (15%). Furthermore, 21% of reviewer self-citations were made without any rationale whatsoever, compared to 5% of citations to others' work [6].

There are three principal forms of peer review: single-blind peer review; double-blind peer review, where both author and reviewer identities are concealed; and open peer review, in which the identities of authors and reviewers are known to one another and, in some settings, reviews are made publically available [8]. It is possible that open review may discourage unnecessary or coercive recommendations by peer reviewers to cite their own work, particularly if reviews are made available to the public [6]. However, no studies have compared the frequency and nature of peer reviewer self-citation in journals with single- or double-blind peer review compared to open peer review.

The objective of this study was to compare peer reviewer self-citation in a journal with single-blind peer review, the Journal of Psychosomatic Research, to a psychiatry journal with a similar impact factor that uses open peer review and makes reviews of published manuscripts available to the public online, BMC Psychiatry. To do this, we compared a subset of reviews of manuscripts that were eventually published in the Journal of Psychosomatic Research from our previous study [6] to reviews of published articles in BMC Psychiatry. We hypothesized that, compared to reviews of manuscripts accepted for publication in the Journal of Psychosomatic Research, reviews of manuscripts accepted for publication in the journal with open peer review, BMC Psychiatry, (1) would have a significantly lower proportion of total citations that were citations to peer reviewers' own work; (2) among self-citations, a greater proportion would include a rationale for the relevance of the citation; and (3) the ratio of the proportion of self-citations without a rationale to the proportion of citations of others' work without a rationale would be lower.

Section snippets

Methods

The coding manual and all methods used in the present study were adapted from our previous study of peer reviewer self-citation [6]. However, because only peer reviews of published articles are provided online by BMC Psychiatry, in the present study we only used data from peer reviews of manuscripts accepted for publication by the Journal of Psychosomatic Research, rather than from all peer reviews, as we did in our previous study.

Statistical analysis

We compared (1) the proportion of citations that were self-citations and (2) the proportion of self-citations that did not include a rationale in the Journal of Psychosomatic Research versus BMC Psychiatry with two-tailed Fisher's exact tests and α = 0.05. We compared the proportions of self-citations without a rationale (out of all self-citations) versus citations to others' work without a rationale (out of all citations to others' work) between the two journals using a z-test for difference in

Results

The Journal of Psychosomatic Research received 298 article submissions in 2012, excluding articles authored by journal editors who were investigators of the previous peer reviewer self-citation study [6] and the present study, as well as articles where all reviews were performed by study investigators. Of these, 155 articles were eventually accepted for publication by the journal. These 155 papers were associated with 334 peer reviews and 304 peer reviewers, where reviews by original study

Discussion

The main findings of this study were that there were no significant or substantive differences between a journal that uses single-blind review, the Journal of Psychosomatic Research, and a generally similar journal that uses open review, BMC Psychiatry, in (1) the proportion of all citations made by peer reviewers that were self-citations, and (2) the proportion of self-citations that included a rationale. In addition, there was no significant difference between the journals in the ratio of the

Conflicts of interest

All authors have completed the Unified Competing Interest form at http://www.icmje.org/coi_disclosure.pdf and declare that no authors have any conflict of interest disclosures for the past 3-year reporting period.

Acknowledgments

Dr. Thombs was supported by an Investigator Salary Award from the Arthritis Society (INS-13-001). There was no specific funding for this study, and no funders had any role in the study design; in the collection, analysis, and interpretation of data; in the writing of the manuscript; or in the decision to submit the manuscript for publication.

References (19)

  • BD Thombs et al.

    Potentially coercive self-citation by peer reviewers: a cross-sectional study

    J. Psychosom. Res.

    (2015)
  • T Jefferson et al.

    Editorial peer review for improving the quality of reports of biomedical studies

    Cochrane Database Syst. Rev.

    (2007)
  • R Smith

    Opening up BMJ peer review: a beginning that should lead to complete transparency

    BMJ

    (1999)
  • M Atkinson

    ‘Peer review’culture

    Sci. Eng. Ethics

    (2001)
  • DB Resnik et al.

    Perceptions of ethical problems with scientific journal peer review: an exploratory study

    Sci. Eng. Ethics

    (2008)
  • B Thombs et al.

    A solution to inappropriate self-citation via peer review

    CMAJ

    (2012)
  • Committee on Publication Ethics

    COPE Ethical guidelines for peer reviewers

  • CJ Lee et al.

    Bias in peer review

    J. Am. Soc. Inf. Sci. Technol.

    (2013)
  • BMC Psychiatry
There are more references available in the full text version of this article.

Cited by (0)

View full text