Year : 2021 | Volume
| Issue : 2 | Page : 179-188
A scoping review of retracted publications in anesthesiology
Marco Fiore1, Aniello Alfieri1, Maria Caterina Pace1, Vittorio Simeon2, Paolo Chiodini2, Sebastiano Leone3, Stefan Wirz4, Arturo Cuomo5, Vincenzo Stoia6, Marco Cascella5
1 Department of Women, Child and General and Specialized Surgery, University of Campania “Luigi Vanvitelli”, Naples, Italy
2 Department of Public, Clinical and Preventive Medicine, Medical Statistics Unit, University of Campania “Luigi Vanvitelli”, Naples, Italy
3 Department of General and Specialized Medicine, “San Giuseppe Moscati” Hospital, Avellino, Italy
4 Abteilung für Anästhesie, Intensivmedizin, Schmerzmedizin/Palliativmedizin - Zentrum für Schmerzmedizin, Weaningzentrum, CURA Krankenhaus, Betriebsstätte der GFO-Kliniken Bonn, Schülgenstr. 15, Bad Honnef, Deutschland, Italy
5 Division of Anesthesia and Pain Medicine, Istituto Nazionale Tumori – IRCCS - Fondazione G. Pascale, Naples, Italy
6 Division of Nuclear Medicine, University of Medicine, “Aldo Moro”, Bari, Italy
Division of Anesthesia and Pain Medicine, Istituto Nazionale Tumori, IRCCS, Fondazione G. Pascale, Via Mariano Semmola, 80131, Naples
Source of Support: None, Conflict of Interest: None
|Date of Submission||14-Nov-2020|
|Date of Decision||14-Nov-2020|
|Date of Acceptance||22-Nov-2020|
|Date of Web Publication||01-Apr-2021|
Context: Fraudulent publication is a scourge of scientific research.
Objectives: This scoping review was aimed at characterizing retracted publications for fraud or plagiarism in the field of anesthesia. Does the reputation of the journal (Quartile and Impact Factor, IF) protect the reader from the risk of having the manuscript he read withdrawn for fraud/plagiarism?
Methods/Design: This scoping review was planned following the Joanna Briggs Institute recommendations. Data sources: PubMed and the Retraction Watch Database (http://retractiondatabase.org/RetractionSearch.aspx?). Study selection: All types of publications retracted. Data extraction: Year, first author nationality, journal name, journal category, IF, Quartile, H index. Data analysis: The association with Quartile and IF was investigated.
Results: No significant association between retraction of papers published in no-Quartile journals and retractions published in journals placed in the highest quartile.
Conclusions: The quality of the surveillance in paper submission is not higher in journals of the first Quartile than in journals not placed in other Quartiles. (The protocol was prospectively registered in the Open Science Framework https://doi.org/10.17605/OSF.IO/TGKNE)
Keywords: Anesthesiology; duplicate publication; fraud; plagiarism; scoping review
|How to cite this article:|
Fiore M, Alfieri A, Pace MC, Simeon V, Chiodini P, Leone S, Wirz S, Cuomo A, Stoia V, Cascella M. A scoping review of retracted publications in anesthesiology. Saudi J Anaesth 2021;15:179-88
|How to cite this URL:|
Fiore M, Alfieri A, Pace MC, Simeon V, Chiodini P, Leone S, Wirz S, Cuomo A, Stoia V, Cascella M. A scoping review of retracted publications in anesthesiology. Saudi J Anaesth [serial online] 2021 [cited 2021 Jun 25];15:179-88. Available from: https://www.saudija.org/text.asp?2021/15/2/179/312954
| Introduction|| |
According to the National Library of Medicine (NLM), Journals may retract articles based on information from their authors, academic or institutional sponsor, editor or publisher, because of pervasive error or unsubstantiated or irreproducible data. Retraction of a scientific paper can broadly be categorized as a result of unintentional or intentional misconduct. Sometimes authors duplicate their data to realize different publications; other times, authorship disputes between co-authors, or between authors and their institution, or any legal concern, can induce the retraction. Because publishing is mandatory to achieve career progression, funding, and prestige for academic institutions and universities, the “publish or perish” paradigm is most appropriate for expressing the publication pressure underlying this regrettable phenomenon., On the other hand, unintentional misconduct can be due to numerous motivations, including mistakes of the publisher (e.g., paper published twice in the same journal, or erroneous issue assignment) or when authors discover and report a serious mistake in their work that invalidates its conclusions. Regardless of the cause, the number of retracted papers is growing rapidly, especially in the fields of medicine and biology. It has been emphasized that about 500-600 scientific papers undergo a retraction process, per year. Of note, from 2001 to 2010, the amount of annual retracted articles, from Open Access Journals, grew by approximately 1000%.
Several studies have been conducted to analyze the retraction phenomenon, to determine its entity in different biomedical fields such as oncology, emergency medicine, drug therapy, radiology. In anesthesiology, a huge number of papers have been retracted although most of these articles were written by only three authors: Yoshitaka Fujii, Joachim Boldt, and Scott Reuben. The “famous” authors Fujii, and Boldt occupy first and second places in the ranking of authors with most retractions in all disciplines. Recently, Dr. Carlisle conducted a statistical analysis on randomized controlled trials (RCTs) published in “anesthesia” and “general medicine” journals in order to evaluate if specific mathematical features (i.e., the mean of continuous variables) of unretracted studies could be associated with a high probability of fraud. Our review was aimed at evaluating qualitative and quantitative features of retracted publications in the field of anesthesia in order to demonstrate that the deleterious impact of the Fujii-Boldt's phenomenon has increased awareness of scientific fraud in anesthesia, inducing, in turn, a substantial improvement in the publication process. Therefore, we associated the quality of the publication process with the percentage of retracted papers, assuming that journals with higher Quartile and higher impact factor (IF) had a more “careful” publication process.
| Methods|| |
The protocol was prospectively registered on 15 May 2019 in the Open Science Framework. It has been planned, according the Joanna Briggs Institute recommendations Scoping Review Methodology Group, and following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Extension for Scoping Reviews (PRISMA-ScR).
This review is designed to answer the following research question:
- Does the reputation of the journal (Quartile and IF) protect the reader from the risk of having the manuscript he read withdrawn for fraud/plagiarism?
This scoping review considered all the retracted publications with no restrictions on the search period (NLM publishes retraction reports, since 1984), language and clinical settings (e.g., elective/emergency anesthesia, pediatric/adult anesthesia). All publication types, preclinical (in vitro/vivo) and clinical researches, editorials, reviews, guidelines, letters, case reports, and case series were included. Papers were excluded if they did not fit into the conceptual framework of the study, focused on the phenomenon of retraction in anesthesia. Moreover, studies presented an anesthesia time but not involving anesthesia protocols, or management, research in anesthesia and related topics were excluded.
We conducted a search query on PubMed using the string “anesthesia AND retract*” and filtering for article type (Retracted Publication). We also perform research on the Retraction Watch Database version: 22.214.171.124 available at http://retractiondatabase.org/RetractionSearch.aspx? [subject: medicine/anesthesia]. Reference lists of relevant studies were also checked. The date of the last search was June 18, 2020.
Manuscript selection, data extraction, and collection
Two authors (A. C. and V. Stoia) independently identified potentially eligible studies, the full text of the retrieved studies was reviewed to select the studies to include in this systematic review. Any disagreement was resolved by consensus with a third reviewer (M. C.). For each article, we recorded the author name, year of publication, the topic of the article, article type (basic, clinical, and research type, as well as papers not involved research), first author's country (affiliation), and year of retraction. In addition, we extracted data on journal name and its metrics, including IF and Quartile, obtained from Journal Citation Report (JCR) 2017. For journal not included in the 'Anesthesiology and Pain Medicine' category and included in more than one category, we considered the best Quartile. The motivations were obtained by the screening of retraction notices released by the journal in which each paper was published. A subsequent analysis was performed to evaluate the scientific impact of each paper through the rate of citations, before and after its retraction (Thomson Scientific's Web of Knowledge).
The percentage of retracted papers was calculated for each journal dividing number of retracted papers by total articles published during the time of observation. For each journal, this parameter was investigated for association with Quartile and IF of the same journal. Journals with no Quartile obtained from JCR were classified as a separate group (No Quartile). Due to highly skewed and not normal distribution data, the associations were tested using a non-parametric test. Kruskal-Wallis test was used to compare the percent of retracted and Quartiles using Dunn's multiple comparisons test for difference among Quartiles. The Spearman rank correlation was used to verify the correlation between the percent of retracted and IF of each journal. A two-tailed P value <0.05 was considered significant. Data were analyzed using R software (version 3.5.0).
| Results|| |
Six hundred seventy-six studied were identified through databases searching (PubMed = 314; Retraction Watch = 362). Four hundred forty-eight papers were screened after removing duplicates. Of these, 21 papers were excluded by title and abstract; consequently, 427 full-text articles were assessed for eligibility. Of these, 4 articles were excluded because journal information not available. Finally, 423 retracted studies were included in the final analysis [Figure 1].
Characteristics of included studies
The list of the 427 retrieved papers and full publishing details can be found in Supplementary material. The journal metrics and number of retractions are reported in [Table 1]; the table synthetizes the journal name and category, the total number of retracted articles and the retraction percentage, the IF, the Quartile, and the H index of the journal and the number of the articles published per year. Almost all the paper retracted, have been retracted for several reasons. These latter were summarized in a table following the strategy used by Marcus and Oransky. [Table 2]. The most common retraction reason is the author misconduct (59,10%) followed by investigation piloted by company/institution (57,21%), the misconduct of an official investigation/finding (50,12%) and the falsification or fabrication of the data (43,74%).
[Figure 2] shows the number of retracted articles for year. The first author of the retracted articles is mainly from Japan and Germany while the nationalities of the journals with greater number of retracted articles are the United States and Great Britain [Figure 3].
|Figure 3: Number of retracted articles for Country (first author and journal)|
Click here to view
In [Figure 4] was reported box plot visualization of the percent of retracted by Quartile subdivision. The Kruskal-Wallis test [H (4) = 16.01, P = 0.003] showed association between the two variable with only the comparison No Quartile vs. 1st Quartile statistically significant [Median (Interquartile range) - No Quartile 0.09 (0.07- 0.67) vs Q1 0.01 (0.006-0.03); P = 0.0055]. In [Figure 5], was reported a scatter plot graph of the percent of retracted and IF. The Spearman's r correlation was -0.4 (P = 0.007) showing a decreasing trend between the percent of retracted and IF.
|Figure 4: Box plot of the percent of retracted by Quartile subdivision. The percent values of retracted were reported as ranks. Only the comparison No Quartile vs. 1st Quartile was statistically significant (P = 0.0055). Legend: Quartile, Q|
Click here to view
|Figure 5: Scatter plot and linear fit of the percent of retracted on y-axis and Impact Factor (IF) on x-axis. Values were log scaled. A decreasing trend between the percent of retracted and IF was observed (P = 0.007). Legend: Quartile, Q|
Click here to view
| Discussion|| |
Although recently, Nair et al. published a comprehensive analysis on the reasons for article retraction in anesthesiology, to our knowledge, this work represents the first scoping review attempting to analyze the phenomenon of scientific retraction in anesthesiology. This phenomenon is easily characterized because three authors – the Fujii-Boldt-Reuben trio – were responsible for about the four-fifths of the retractions. Dr Yoshitaka Fujii was at the center of a famous editorial case. Fujii and co-authors 'conducted' a huge number of investigations to dissect all the aspects of postoperative nausea and vomiting (PONV). Data were published on prestigious anesthesia and non-anesthesia journals, and researchers began to doubt on their sincerity. The Japanese Society of Anesthesiologists Special Investigation Committee on Fujii's Papers confirmed that an incredible number of articles were fabricated and only 3 papers were verified as authentic. The second striking case is the Boldt affair., Between the beginnings 1990s and 2010 the German anesthesiologist published numerous articles on fluid management (mainly on hydroxyethyl starch, HES). Initially, a retraction of 88 Boldt's publications was due to lack of ethics approval. Subsequently, Boldt was suspicioned about design and data classification, as well as data authenticity. For instance, although Boldt affirmed to use albumin in his studies on cardiac surgery, the Klinikum Ludwigshafen (Boldt's employer) stated that no albumin was used in that setting, since 1999. The last case regards the American Scott Reuben. In 2009, a notice of retraction the editorial office of the journal Anesthesia & Analgesia notified that 10 Reuben's articles were retracted for fabricated data. In the same year, there was the retraction of others 21 articles published between 1996 and 2008.
The fraudulent conduct of these three authors has also influenced the country analysis and the temporal trend of the retraction phenomenon, because the trio have acted above all in the 1989-2008 period and after their unravelling, the trend has almost halved. Furthermore, many recent retraction reports refer to their studies published more than ten years before. Apparently, in absence of qualified editor section for the matter it can be easier for fraud to remain misunderstood. On the other hand, very important journals including some of the JAMA Network family were involved in the fraud. It is of note, however, that the most important journals with the greater IFs have not a reduced percentage of retraction, and this finding could be explained by a more a greater number of readers and therefore a greater possibility to find criticisms in published papers. In fact, our analysis showed no significant association in retraction between the journals with “No Quartile” vs journals with the highest Quartile with a decreasing trend between the percent of retracted and higher IF.
Fraud and plagiarism are the main reasons for retraction. Furthermore, about a quarter of all the articles were retracted due to ethical problems. However, numerous Boldt's papers initially removed for ethical issue also presented altered data.
The matter of motivations for retraction is rather complex and further clarification from the scientific community should be carried out. In particular, because the word “retraction” can represent a stain in the career of a researcher, problems on fabrication or falsification of data, plagiarism, and ethical issues in research should be differentiated from other circumstances in which the retraction has been induced by an administrative error of problems due to editing process. Probably, in these latter conditions it should be more appropriate to indicate the paper as “withdrawal”. Many publishers already use various terms for notices. For example, some adopt “removal” or “retraction” when the retraction is initiated by the editors, and “withdrawal” when it is initiated by the authors. Others use “retraction” uniformly and still others use “withdrawal”. Moreover, other publishers label all notations as “errata”. Thus, a uniform nomenclature seems to be needed.
How to easily detect scientific fraud? A mathematical model was used by Kranke et al. to launch a warning on Fujii's studies reliability; a similar strategy was adopted for investigating on 3 biochemical researchers. As previously mentioned, Dr Carlisle used the Stouffer's method to detect anomalies in the distributions of baseline continuous variables reported as mean to evaluate possible frauds in unretracted RCTs in anesthesiology. It was the same approach used to investigate on the data integrity of the Fujii's studies. The Stouffer's method was used to combine the P values of multiple variables. After calculation of about 30.000 variables, Carlisle found that RCTs with extreme distributions of means were far more suspicious of containing fraud data than other studies. In other words, when P values are so extreme it is very likely that the baseline data are fabricated.
The meta-analyses issue. A tremendous bias that is dragging on is that Boldt's studies continued to be included in meta-analyses after retraction. The same problem also regards the Fujii's studies. For instance, a Cochrane analysis on PONV have included data from Fujii's 'investigations'. However, Dr. Carlisle performed a newest meta-analysis on PONV comparing findings from Fujii's trials with those of other authors. As a consequence, including fraudulent data in a meta-analysis substantially prejudices the results and meta-analysts should carefully consider this bias.
Strengths and limitations
Our analysis has several limitations. For example, the journals metric refers to 2017 data. However, the analysis started before the new indices were released (2020).
The research methodology certainly has several limitations. Retractions and retracted publications are not always properly crossed linked. Several papers are even indexed as corrections and can be indicated as “correction and republished article” and as “published erratum”. Following the screening of the articles, many of these possible sources of bias were identified. Additionally, other important databases such as Web of Science, J-STAGE, and KoreaMed, also index retractions in anesthesia, and those journals are not all indexed in PubMed. Nevertheless, expanding the search to other databases would have taken us far from the scope of this review that was aimed at assessing the association between the journal's reputation and retraction for fraud or plagiarism.
Another important limitation concerns the lack of data on the number of articles accepted or rejected by Q1 journals. Although the knowledge of these data would have provided us with a greater awareness of the phenomenon, such an exhaustive analysis would have considerably complicated the study, taking us outside the main purpose.
It would have been interesting to evaluate the retraction phenomenon by referring to the date of the first suspicions on Fujii's publications and to evaluate the trend of the retractions before and after. The great limitation of the analysis is in the very nature of the phenomenon. Of note, after the completion of the research, a lot of new retractions have been released. We considered it appropriate not to include the new data in the analysis because the real purpose of the publication was to underline that: (i) retraction is not associated with the journal's reputation; (ii) in addition to the ability of editors and reviewers, dedicated software can help unmask fraud; (iii) the term retracted (e.g., retraction note or retraction notice) should be reserved for true fraud, while for articles canceled for non-fraudulent causes, journals should use the term “withdrawn”.
| Conclusions|| |
Our analysis showed no association in retraction between the journals with “No Quartile” vs journals with the “1st Quartile” with a no significant decreasing trend between the percent of retracted and higher IF. Therefore, a careful publication process seems to reduce drastically the acceptance of fraudulent papers. In our opinion, an aspect that should be re-evaluated is the large citation of retracted articles and their use in meta-analysis.
M. F. and A. A. designed the study. A. C., V. Stoia, and M. C. contributed to the literature search, data extraction, and data analysis. M. C., S. W., M. C. P, and S. L. contributed to the project design and writing of the manuscript. V. S., and P. C. performed the statistical analysis. All authors have read and approved the final version of the manuscript. All authors made substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; took part in drafting the article or revising it critically for important intellectual content; gave final approval of the version to be published; and agree to be accountable for all aspects of the work.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Lu SF, Jin GZ, Uzzi B, Jones B. The retraction penalty: Evidence from the Web of Science. Sci Rep 2013;3:3146.
Neema PK. Medical research: Is everything all right? J Anaesthesiol Clin Pharmacol 2011;27:159-61.
] [Full text]
Van Noorden R. Science publishing: The trouble with retractions. Nature 2011;478:26-8.
Wang T, Xing QR, Wang H, Chen W. Retracted publications in the biomedical literature from open access journals. Sci Eng Ethics 2019;25:855-68.
Bozzo A, Bali K, Evaniew N, Ghert M. Retractions in cancer research: A systematic survey. Res Integr Peer Rev 2017;2:5.
Chauvin A, De Villelongue C, Pateron D, Yordanov Y. A systematic review of retracted publications in emergency medicine. Eur J Emerg Med 2019;26:19-23.
Samp JC, Schumock GT, Pickard AS. Retracted publications in the drug literature. Pharmacotherapy 2012;32:586-95.
Rosenkrantz AB. Retracted publications within radiology journals. AJR Am J Roentgenol 2016;206:231-5.
Carlisle JB. Data fabrication and other reasons for non-random sampling in 5087 randomised, controlled trials in anaesthetic and general medical journals. Anaesthesia 2017;72:944-52.
Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, et al
. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth 2020;18:2119-26.
Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al
. PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Ann Intern Med 2018;169:467–73.
Nair S, Yean C, Yoo J, Leff J, Delphin E, Adams DC. Reasons for article retraction in anesthesiology: A comprehensive analysis. Can J Anaesth 2020;67:57-63.
Tramèr MR. The Fujii story: A chronicle of naive disbelief. Eur J Anaesthesiol 2013;30:195-8.
Kranke P, Apfel CC, Roewer N, Fujii Y. Reported data on granisetron and postoperative nausea and vomiting by Fujii et al
. Are incredibly nice! Anesth Analg 2000;90:1004-7.
Miller DR. Retraction of articles written by Dr. Yoshitaka Fujii. Can J Anesth 2012;59:1081-8.
Farrugia A. The Boldt affair: Correcting a collective failure. Anesth Analg 2012;115:207.
Tramer MR. The Boldt debacle. Eur J Anaesthesiol 2011;28:393-5.
Editors-in-Chief statement regarding published clinical trials conducted without IRB approval by Joachim Boldt. Minerva Anestesiol 2011;77:562-3.
Shafer SL. Notice of retraction. Anesth Analg 2009;108:1350.
Eisenach JC. Data fabrication and article retraction: How not to get lost in the woods. Anesthesiology 2009;110:955-6.
Hudes ML, McCann JC, Ames BN. Unusual clustering of coefficients of variation in published articles from a medical biochemistry department in India. FASEB J 2009;23:689-703.
Carlisle JB. The analysis of 168 randomised controlled trials to test data integrity. Anaesthesia 2012;67:521-37.
Bunn F, Trivedi D. Colloid solutions for fluid resuscitation. Cochrane Database Syst Rev 2012;7:CD001319.
Carlisle JB, Stevenson CA. Drugs for preventing postoperative nausea and vomiting. Cochrane Database Syst Rev 2006;2006:CD004125.
Carlisle JB. A meta-analysis of prevention of postoperative nausea and vomiting: Randomised controlled trials by Fujii et al
. compared with other authors. Anaesthesia 2012;67:1076-90.
Wilkes MM, Navickis RJ. The Boldt affair: A quandary for meta-analysts. Anesthesiol News 2013;39:8-9.
Hemmings HC Jr, Shafer SL. Further retractions of articles by Joachim Boldt. Br J Anaesth 2020;125:409-11.
[Figure 1], [Figure 2], [Figure 3], [Figure 4], [Figure 5]
[Table 1], [Table 2]