|Year : 2018 | Volume
| Issue : 4 | Page : 294-297
Do dental research journals publish only positive results? A retrospective assessment of publication bias
Praveen Gadde1, Gautami Subhadra Penmetsa2, Keerthana Rayalla2
1 Department of Public Health Dentistry, Vishnu Dental College, Bhimavaram, Andhra Pradesh, India
2 Department of Periodontics and Implantology, Vishnu Dental College, Bhimavaram, Andhra Pradesh, India
|Date of Submission||24-Jan-2018|
|Date of Acceptance||07-May-2018|
|Date of Web Publication||20-Jul-2018|
Dr. Praveen Gadde
Department of Public Health Dentistry, Vishnu Dental College, Vishnupur, Bhimavaram, West Godavari - 534 202, Andhra Pradesh
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Background: The study aimed to determine the presence of publication bias in the top five dental journals with high impact factor published during 2007–2016. Materials and Methods: The journals included are Journal of Clinical Periodontology, Dental Materials, Clinical Oral Implant Research (COIR), Journal of Endodontics, and Journal of Dental Research (JDR). The content of these journals was hand searched by two authors for scouring the proportion of positive or negative results. Articles showing two outcomes (primary and secondary) were registered depending on the primary result. Results: The present study revealed higher percentage of statistically significant results in the published dental literature. Of five journals, JDR has shown fewer tendencies toward publication of nonsignificant results, whereas COIR has evidenced the highest publication of nonsignificant results. The journals with higher impact factor showed significantly different acceptance rate for research with positive results. However, year-wise publication (2007–2016) did not depict any significant difference. Conclusion: In dental research journals, articles with positive results are finding a better way in getting published compared to articles with negative results.
Keywords: Clinical trials, dental journals, impact factor, positive results, publication bias
|How to cite this article:|
Gadde P, Penmetsa GS, Rayalla K. Do dental research journals publish only positive results? A retrospective assessment of publication bias. J Indian Soc Periodontol 2018;22:294-7
|How to cite this URL:|
Gadde P, Penmetsa GS, Rayalla K. Do dental research journals publish only positive results? A retrospective assessment of publication bias. J Indian Soc Periodontol [serial online] 2018 [cited 2020 Feb 23];22:294-7. Available from: http://www.jisponline.com/text.asp?2018/22/4/294/237207
| Introduction|| |
Progress in the life sciences including medicine and dentistry is heavily dependent on the data obtained from research and subsequent publications. However, it has been well recognized that a great publication bias exists toward the publications that endorse previously published material or author's own initial hypothesis. Nonendorsement of already available data or nonrejection of the null hypothesis makes negative data. Articles with negative data rarely get a chance for publication. Hence, most of the authors feel intimidated to produce articles which carefully avoid negative data.
Experienced researchers are aware of this bias and seldom venture into contradicting the popular data in an attempt to ensure publication in reputed journals. Rarely, this norm may be challenged by publishing all the results irrespective of their outcomes. Many authors believe that the null hypothesis they have raised has to be rejected failing which getting a publication may be a foregone dream.
Dental researchers believe that the negative data have no value and all the journals maintain that attitude. They are also well aware of the fact that editors exhibit less interest in publishing studies that do not depict the previously available data. An investigation was carried out at different countries which included 4600 publications showing a great increase in publication bias for the past few years. There was almost an increased proportion of publishing significant results by 22% from 1990 to 2007. Although the topic of publication bias has been evidenced in the literature for decades and its cause and effects have been discussed extensively, there is no established evidence in dental literature.
Hence, the authors of this article wanted to evaluate this fact after examining the articles published in five reputed journals with reasonably good impact factor.
| Materials and Methods|| |
The following five dental research journals with highest impact factor in the past 5 years (as per the data of 2016) were considered in the study:
- Journal of Clinical Periodontology (JCP)
- Dental Materials (DMs)
- Clinical Oral Implant Research (COIR)
- Journal of Endodontics (JOE)
- Journal of Dental Research (JDR).
On an average, the impact factor of journals included was as follows: JCP: 2.05–4.47, DM: 1.07–2.28, COIR: 2.54–3.43, JOE: 3.09–4.47, and JDR: 3.09–4.47. The content of the above-mentioned dental journals over the past decade was hand searched by the authors for scouring the proportion of published positive/negative results between January 2007 and December 2016. The journal search was performed between January 2016 and May 2016 in the Central Library of Vishnu Dental College. An article is considered to have negative or nonsignificant results where the investigators were unable to reject their null hypothesis and a tool of frequentist statistics which states that there is no difference between experimental groups.
All the original articles published were analyzed for the presence of significant or nonsignificant results. The articles without statistical analysis, solitary case reports, review articles, position papers, letters to editors, etc., were excluded from the study. Articles comparing newer techniques or surgical procedures without proper statistical comparison were not considered in the analysis. In case where an article showed two contradictory outcomes (primary and secondary), the article outcome was taken into consideration based on the primary outcome.
A total of 924 articles among 130 issues published in the JCP, 1130 articles among 120 issues from the DM journal, 848 articles from 120 issues that belonged to COIR, 120 issues with 968 articles from JOE, and 120 issues with 799 articles from JDR were physically searched, while on the other hand, 164 articles of JCP, 179 articles of COIR, 88 articles of JOE, and 100 articles of JDR where the printed copy was not available were searched electronically, thereby ensuring that none of the studies were missed.
Altogether 1088 articles in JCP, 1130 articles in DMs, 1027 articles in COIR, 1056 articles in JOE, and 899 articles in JDR were analyzed, thereby making up a sum of 5200. In total, 5608 articles were analyzed for the study, and 408 articles were excluded for not satisfying the above-mentioned criteria.
Taking statistical analysis into consideration, the descriptive statistics on the characteristics of the articles were calculated using univariate and multivariate logistic regressions. These duo analyses were used to examine the association between articles reporting a statistically significant result or the one rejecting the null hypothesis (dependent variable) and journal impact factor, subject area, and year of publication (independent variables). All the analyses were conducted with statistical software IBM SPSS Statistics for Windows, version 21.0. Armonk, NY, USA: IBM Corp. The statistical significance was set at 0.05 for all the comparisons.
| Results|| |
[Table 1] shows the distribution of 5200 articles by specialty, impact factor, year of publication, and significance of the primary findings. The data indicate that there was a statistical significant difference identified with respect to subject area and impact factor publishing positive results, with no statistical significant difference in the year of publication. These journals showed significantly varying publishing rates of results that were positive ranging from 86.17% to 97.11% (P = 0.0001). JDR has reported increased proportion of publishing positive results (97.11%) whereas COIR journal reported highest number negative results (13.83%) published compared to other subject journals.
|Table 1: Distribution of articles by subject area, year of publication, impact factor, and main findings level of statistical significance|
Click here to view
When impact factor was analyzed, journals with impact factor >4.0 reported high proportion of positive results (90.87%) compared to journals with impact factor < 4.0 (P = 0.004). There was no statistical significant difference identified in publishing positive/negative results with respect to the year of publication from 2007 to 2016 (P = 0.4301).
In [Table 2], the odds ratios derived from the univariate and multiple logistic regression analyses and confidence intervals are given. This analysis revealed that compared to JCP, all other subject journals showed decreased probability of publishing positive results ranging from 24% to 89%. The finding was statistically significant for all the journals. When compared journals with impact factor >4.0, other journals have showed decreased probability of publishing positive results. With the year of publication also, there was a decreased probability of publishing positive results compared to remaining periods of publication.
|Table 2: Univariate and multivariate logistic regression derived odds ratio and confidence intervals of journal characteristics|
Click here to view
| Discussion|| |
Science is not a static entity and is a never-ending and dynamic subject. As it is subjected to continuous change, it is the need of the hour for every clinician and researcher to get upgraded with the knowledge. Health-care researchers can access evidence from a variety of sources for clinical decision-making. The most powerful and increasingly used analytic tool in dental research is the systematic review, particularly those employing meta-analysis. These tools, in turn, depend on the validity of already published data. The selective publication of positive or significant results against negative or nonsignificant results is so distressing because it can distort a picture of gathered evidence. The results of this investigation showed higher proportion of publishing positive results in dental research compared with negative results which is in the same trend that is seen in medical research. The similar articles with two different results were found to differ by almost 40% in their acceptance rate; on the other hand, the articles with nonsignificant results were accepted less often.
Several authors have discussed various factors that contribute to publication bias. A common reason for failure to publish positive results is “self-censoring of trial results.” Majority of researchers have a strong opinion that negative findings carry a low priority for publication. The work of Dickerson suggests that rejection by journal editors and reviewers of studies with negative results is also one major contribution to publication bias in dental research. An article with nonsignificant result that depicts the outcome of a treatment technique will have a better way to be published because it deals with current situation. On the other hand, a study reporting a nonsignificant result of a newer technique is judged to have no importance.
Manuscripts with positive results are more likely to be written and submitted by investigators and sponsors for publication. Peer reviewers and editors generally access the articles with positive results. The publication bias not necessarily depends on journal impact factor but as per the New England Journal of Medicine 2016 in which it was stated that “It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines.” On the other hand, nonsignificant results are shown to be less important, which, in turn, were of at any interest in attracting readers and citations. Hence, a journal with high impact factor will not be helpful in getting published.
The negative impact on health care has been substantially quantified and this needs to be critically evaluated, and in this aspect, the effect of publication bias on dental research is very critical. As there is a failure of publishing negative results, the possibility of developing a biased pool of evidence research exists. The biased conclusion in systematic reviews and the point estimate of effect in meta-analysis could be a result of an analysis of the evidence.
This could be very well explained by an example of a research that looked into the effect of combination chemotherapy versus a single agent in the treatment of ovarian cancer. Favorable evidence toward the combination therapy was put forth by unregistered published trials. However, on the other hand, a subsequent analysis of both the published and unpublished data revealed no appreciable benefit to combination therapy. On the other hand, research involving human or animal subjects has been deemed as a form of scientific misconduct, especially when there is a failure in publishing the results. Moreover, consideration does also exist when resources on a research project have been carried out and utilized for publication. One of the most important things that need to be taken into consideration is that publication of a good research can prevent the duplication of efforts despite the negative findings incurred in the study.
The reversing of negativity toward negative results has sparked a movement resulting in a revolt against publication bias. Combating this problem, on the 14th of April 2015, WHO made a press statement that the findings of clinical trials have to be published through an open access mechanism unless there is a specific reason why open access cannot be used or otherwise made available publicly at most within 24months of study completion. Journal editors and reviewers should avoid in seeking manuscripts with positive results. One of the vital concerns today regarding the critical trial databases is the difficulty in reporting research findings, especially those funded by the sponsors; therefore, the data are often not available for open access.,
| Conclusion|| |
The present study revealed that the proportion of publishing positive results is higher among the dental journals compared to negative results. On the other hand, as a result of publication bias as in medicine, systematic reviews and meta-analysis will draw negative conclusions. Hence, the awareness toward the potential problem arising from publication bias and the need to fight against it should be one of the sole responsibilities of the dental researcher. Hence, considering the quality of the research done rather than the result of the study in publishing the article should be the prime criteria.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Koletsi D, Karagianni A, Pandis N, Makou M, Polychronopoulou A, Eliades T, et al.
Are studies reporting significant results more likely to be published? Am J Orthod Dentofacial Orthop 2009;136:632.e1-5.
Joober R, Schmitz N, Annable L, Boksa P. Publication bias: What are the challenges and can they be overcome? J Psychiatry Neurosci 2012;37:149-52.
Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, Cronin E, et al.
Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One 2008;3:e3081.
Egger M, Smith DG, O'Rourke K. Rationale, Potentials, and Promise in Systematic Reviews. London, United Kingdom: British Medical Journal Publishing Group; 2001. p. 3-19.
Praveen G, Anitha A, Ramesh MV. Negating the negativity: Spotlight on “file drawer effect” in health care research. J Indian Assoc Public Health Dent 2016;14:243. [Full text]
Polychronopoulou A, Pandis N, Eliades T. Assessment of publication bias in dental specialty journals. J Evid Based Dent Pract 2010;10:207-11.
Radford DR, Smillie L, Wilson RF, Grace AM. The criteria used by editors of scientific dental journals in the assessment of manuscripts submitted for publication. Br Dent J 1999;187:376-9.
Sandhya J, Sharmila D, Deshraj J. Bias in dental research/dentistry. Ann Int Med Dent Res 2016;2:5-9.
Marcia A. No longer possible to believe much of clinical research published. N
Engl J Med 2009;361:1-14.
Scholey JM, Harrison JE. Publication bias: Raising awareness of a potential problem in dental research. Br Dent J 2003;194:235-7.
Simes RJ. Publication bias: The case for an international registry of clinical trials. J Clin Oncol 1986;4:1529-41.
Chalmers I. Underreporting research is scientific misconduct. JAMA 1990;263:1405-8.
Dirnagl U, Lauritzen M. Fighting publication bias: Introducing the negative results section. J Cereb Blood Flow Metab 2010;30:1263-4.
[Table 1], [Table 2]