Promotional language in radiology publications: increasing use of “excellent”, “favorable”, “promising”, “robust”, and “unique”

Keywords

bias - communication - medical imaging - publishing - radiology

Abbreviations

NC: not calculable (due to the fact that the percentage of articles with the word “phenomenal” in the title/abstract was 0 % in each year between 2003 and 2022)

Introduction

Scientific output, both in terms of quantity and impact, is considered a major criterion for academic employment, promotions, and prestige [1]. Publishing in prestigious journals is frequently a target for researchers. However, journals, and particularly those with a high impact factor (IF), only accept a minority of submitted manuscripts. Not surprisingly, publication pressure on academic researchers is generally regarded as high [2] [3] [4].

Too much publication pressure may decrease the quality of the science when the aim of researchers shifts to producing “publishable” results at the expense of performing high-quality and meaningful research [2] [3] [4]. This is further aggravated by the fact that studies with positive results are more likely to be published and to be published faster than those with negative results [5] [6].

Publication pressure and publication bias may also affect the way in which scientists communicate their results in scientific publications. A previous publication in the British Medical Journal showed that the absolute frequency of positive words (such as “robust,” “novel,” “innovative,” and “unprecedented”) in scientific abstracts in PubMed increased from 2.0 % (1974–1980) to 17.5 % (2014), which translates to a relative increase of 775 % over four decades [7]. It has been postulated that scientists assume that results and their implications have to be exaggerated and overstated in order to get published [7]. Whether or not this phenomenon also plays a role in the field of radiology is still unclear. This knowledge is important to understand how the text of published radiology research should be interpreted by the readers of these articles. We hypothesized that the frequency of positive words in the radiological literature has increased over the past two decades, and that this temporal pattern can be observed throughout the entire range of journal IFs.

The purpose of this study was therefore to investigate if radiology researchers are increasingly promoting their scientific findings by more frequently using positive words in their publications.

Materials and Methods

Study selection

This study included all articles that were published in 14 general radiology journals (Radiology, Investigative Radiology, Korean Journal of Radiology, European Radiology, American Journal of Roentgenology, Radiologia Medica, Journal of the American College of Radiology, Academic Radiology, European Journal of Radiology, Canadian Association of Radiologists Journal, British Journal of Radiology, Clinical Radiology, Clinical Imaging, and Acta Radiologica, with IFs ranging between 29.146 and 1.701 [8]) between January 1, 2003 and December 31, 2022. The selection of these journals was based on their scope (general radiology rather than a subspecialty field) and IF. Ethical review board approval and informed consent were not required for this bibliometric analysis.

Data extraction

The title and abstract of each article were assessed for the presence of any positive word, any negative word, any neutral word, and any random word, according to predefined sets of words for each category. These sets of words were adapted from a previous study [7] and are displayed in [Table 1]. The analysis of each abstract for the presence of any positive word, any negative word, any neutral word, and any random word was done using combinations of different search strings in PubMed, which are displayed in Supplementary Table 1. The publication year of each article and the IF of the journal in which it was published were manually extracted.

Table 1 Predefined sets of 25 words for each category (positive words, negative words, neutral words, and random words) adapted from [7].

Category	Words
Positive words (n = 25)	Amazing, Assuring, Astonishing, Bright, Creative, Encouraging, Enormous, Excellent, Favorable, Groundbreaking, Hopeful, Innovative, Inspiring, Inventive, Novel, Phenomenal, Prominent, Promising, Reassuring, Remarkable, Robust, Spectacular, Supportive, Unique, Unprecedented
Negative words (n = 25)	Detrimental, Disappointing, Disconcerting, Discouraging, Disheartening, Disturbing, Frustrating, Futile, Hopeless, Impossible, Inadequate, Ineffective, Insignificant, Insufficient, Irrelevant, Mediocre, Pessimistic, Substandard, Unacceptable, Unpromising, Unsatisfactory, Unsatisfying, Useless, Weak, Worrisome
Neutral words (n = 25)	Animal, Blood, Bone, Brain, Condition, Design, Disease, Experiment, Human, Intervention, Kidney, Liver, Man, Men, Muscle, Patient, Prospective, Rodent, Significant, Skin, Skull, Treatment, Vessel, Woman, Women
Random words (n = 100)	Manager, Substance, Law, Dust, Bite, Butter, Fold, Mind, Protect, Insurance, Test, Father, Letter, Friend, Power, Edge, Linen, Scale, Bread, Statement, Weather, Smell, Glass, Food, Level, Steam, Soap, Help, Rule, Wind, Interest, Purpose, Hole, Fight, Representative, Danger, Prose, Change, Discussion, Company, Direction, Balance, Organization, Size, Trade, Rice, Invention, Heat, Road, Mountain, Electric, Good, Natural, Sweet, Dead, Strange, Thin, Political, Open, Bitter, Dark, Complex, Warm, Full, Red, Kind, Possible, Strong, Free, Quick, Slow, Cut, Narrow, Certain, Dependent, Flat, Acid, Fixed, Responsible, False, Great, Like, Green, Cold, Poor, Low, Opposite, Bright, Military, Fertile, Second, Left, Wrong, Hanging, Gray, Mixed, Angry, Foolish, Loose, Late

Data analysis

The number of articles with any positive word in the title/abstract was divided by the total number of articles for all 14 journals together and for each year between 2003 and 2022 separately. In this way, “positive word usage” was calculated for each year. Similarly, negative word usage, neutral word usage, and random word usage were calculated for each year. Mann-Kendall tests were done to test for any temporal changes in word usage between 2003 and 2022. Temporal analyses were also done for each individual positive word (i. e., “Amazing”, “Assuring”, “Astonishing”, “Bright”, “Creative”, “Encouraging”, “Enormous”, “Excellent”, “Favorable”, “Groundbreaking”, “Hopeful”, “Innovative”, “Inspiring”, “Inventive”, “Novel”, “Phenomenal”, “Prominent”, “Promising”, “Reassuring”, “Remarkable”, “Robust”, “Spectacular”, “Supportive”, “Unique”, and “Unprecedented”). Only those positive words that showed significantly increased usage over time and that were present in more than 1 % of titles/abstracts in at least one year were considered relevant. Subsequently, a linear regression analysis was performed to assess the association between the presence of any positive word in the title/abstract and journal IF. P-values less than 0.05 were considered statistically significant. Statistical analyses were performed using R version 4.2.2 software (R Foundation for Statistical Computing) and MedCalc Statistical Software version 18.5 (MedCalc, Ostend, Belgium).

Results

Articles

A total of 94,550 articles were published in the 14 included journals between 2003 and 2022. The total annual number of articles, and annual numbers of articles with at least one positive word, negative word, neutral word, and random word in the title/abstract between 2003 and 2022 are displayed in [Table 2].

Table 2 Total number of articles (with percentage increase or decrease compared to the year 2003 between parentheses), and number of articles with at least one positive word, negative word, neutral word, and random word (according to predefined sets of words for each category as shown in [Table 1]) in the title or abstract for each year between 2003 and 2022.

Year	Total no. of articles (with percentage increase or decrease compared to the year 2003 between parentheses)	No. of articles with at least one positive word in the title or abstract	No. of articles with at least one negative word in the title or abstract	No. of articles with at least one neutral word in the title or abstract	No. of articles with at least one random word in the title or abstract
2003	3035	206	61	1770	1764
2004	3331 (+ 9 %)	200	59	1930	1987
2005	3580 (+ 18 %)	258	77	2131	2248
2006	3590 (+ 18 %)	317	67	2265	2394
2007	3679 (+ 21 %)	321	80	2344	2433
2008	3896 (+ 28 %)	357	88	2569	2694
2009	4047 (+ 33 %)	412	107	2772	2898
2010	4138 (+ 36 %)	445	84	2814	2988
2011	4747 (+ 56 %)	519	93	3374	3552
2012	5064 (+ 67 %)	510	109	3531	3708
2013	4663 (+ 54 %)	505	96	3178	3383
2014	4525 (+ 49 %)	473	83	3154	3363
2015	5098 (+ 68 %)	568	126	3553	3749
2016	5231 (+ 72 %)	657	136	3629	3835
2017	5113 (+ 68 %)	645	113	3594	3751
2018	5611 (+ 85 %)	707	134	3898	4080
2019	5519 (+ 82 %)	733	139	3828	3941
2020	6677 (+ 120 %)	965	146	4724	4748
2021	6948 (+ 129 %)	1011	148	4785	4889
2022	6058 (+ 99 %)	910	126	4099	4177

Temporal trends in the use of positive, negative, neutral, and random words

Positive word usage (Mann-Kendall tau of 0.895, P< 0.001) and neutral word usage (Mann-Kendall tau of 0.463, P = 0.005) showed significant upward temporal trends. The annual frequency of positive words increased from 7 % in 2003 to 15 % in 2022 (relative increase of 120 %), and the annual frequency of neutral words increased from 58 % in 2003 to 68 % in 2022 (relative increase of 16 %). Negative word usage (Mann-Kendall tau of 0.242, P = 0.1443) and random word usage (Mann-Kendall tau of 0.305, P = 0.064) did not show any significant temporal trends. Temporal trends are visually displayed in [Fig. 1], [2], [3], [4].

Fig. 1 Graph with percentages of articles with any positive word (according to a predefined set of words as shown in Table 1) in the title/abstract between 2003 and 2022. The gray line denotes the proportions (%) per year and the blue line represents the non-parametric locally estimated scatterplot smoothing (LOESS) fit in each graph. Positive word usage showed a significant upward temporal trend (Mann-Kendall tau of 0.895, P< 0.001).

Fig. 2 Graph with percentages of articles with any negative word (according to a predefined set of words as shown in Table 1) in the title/abstract between 2003 and 2022. The gray line denotes the proportions (%) per year and the blue line represents the non-parametric locally estimated scatterplot smoothing (LOESS) fit in each graph. Negative word usage did not show any significant temporal trend (Mann-Kendall tau of 0.242, P = 0.1443).

Fig. 3 Graph with percentages of articles with any neutral word (according to a predefined set of words as shown in Table 1) in the title/abstract between 2003 and 2022. The gray line denotes the proportions (%) per year and the blue line represents the non-parametric locally estimated scatterplot smoothing (LOESS) fit in each graph. Neutral word usage showed a significant upward temporal trend (Mann-Kendall tau of 0.463, P = 0.005).

Fig. 4 Graph with percentages of articles with any random word (according to a predefined set of words as shown in Table 1) in the title/abstract between 2003 and 2022. The gray line denotes the proportions (%) per year and the blue line represents the non-parametric locally estimated scatterplot smoothing (LOESS) fit in each graph. Random word usage did not show any significant temporal trend (Mann-Kendall tau of 0.305, P = 0.064).

Temporal trends in the use of individual positive words

Five positive words showed significantly increased usage over time and were present in more than 1 % of titles/abstracts in at least one year: “excellent” (Mann-Kendall tau of 0.800, P< 0.001), “favorable” (Mann-Kendall tau of 0.547, P< 0.001), “promising” (Mann-Kendall tau of 0.607, P< 0.001), “robust” (Mann-Kendall tau of 0.737, P< 0.001), and “unique” (Mann-Kendall tau of 0.747, P< 0.001) ([Table 3]). Temporal trends are visually displayed in [Fig. 5], [6], [7], [8], [9].

Table 3 Temporal trend analyses for the use of individual positive words in the title/abstract in 14 general radiology journals between 2003 and 2022. Mann-Kendall tau and corresponding P-values are shown. The percentages of articles with a positive word in the title/abstract in the years 2003 and 2022 are also shown. Note that the absolute number of articles increased from 3035 in the year 2003 to 6058 in the year 2022 (99 % increase). Individual positive words that showed significantly increased usage over time and that were present in more than 1 % of titles/abstracts in at least one year between 2002 and 2022 are marked in bold italics.

Positive word	Mann-Kendall tau	P-value	Percentages of articles with a positive word in the title or abstract
			2003	2022
Amazing	0.417	0.032	0 %	0.017 %
Assuring	0.288	0.116	0 %	0.017 %
Astonishing	0.370	0.059	0 %	0.017 %
Bright	–0.400	0.015	0.329 %	0.116 %
Creative	0.341	0.044	0 %	0.066 %
Encouraging	0.216	0.194	0.099 %	0.231 %
Enormous	–0.069	0.697	0 %	0.066 %
Excellent	0.800	< 0.001	2.405 %	4.738 %
Favorable	0.547	< 0.001	0.264 %	1.255 %
Groundbreaking	0.102	0.606	0 %	0 %
Hopeful	0.087	0.674	0 %	0 %
Innovative	0.589	< 0.001	0.066 %	0.314 %
Inspiring	0.368	0.043	0 %	0.050 %
Inventive	0.0166	1.000	0 %	0 %
Novel	0.872	< 0.001	0.008 %	0.037 %
Phenomenal	NC	NC	0 %	0 %
Prominent	–0.232	0.163	0.626 %	0.248 %
Promising	0.607	< 0.001	1.087 %	3.021 %
Reassuring	–0.101	0.572	0 %	0.017 %
Remarkable	0.317	0.055	0.165 %	0.198 %
Robust	0.737	< 0.001	0.395 %	1.403 %
Spectacular	–0.156	0.426	0 %	0 %
Supportive	0.550	< 0.001	0.099 %	0.165 %
Unique	0.747	< 0.001	0.659 %	1.139 %
Unprecedented	0.000	1.000	0.165 %	0.066 %

Fig. 5 Graph with percentages of articles with the positive word “excellent” in the title/abstract between 2003 and 2022. The gray line denotes the proportions (%) per year and the blue line represents the non-parametric locally estimated scatterplot smoothing (LOESS) fit in each graph. Usage of “excellent” showed a significant upward temporal trend (Mann-Kendall tau of 0.800, P< 0.001).

Fig. 6 Graph with proportions of articles with the positive word “favorable” in the title/abstract between 2003 and 2022. The gray line denotes the proportions (%) per year and the blue line represents the non-parametric locally estimated scatterplot smoothing (LOESS) fit in each graph. Usage of “favorable” showed a significant upward temporal trend (Mann-Kendall tau of 0.547, P< 0.001).

Fig. 7 Graph with percentages of articles with the positive word “promising” in the title/abstract between 2003 and 2022. The gray line denotes the proportions (%) per year and the blue line represents the non-parametric locally estimated scatterplot smoothing (LOESS) fit in each graph. Usage of “promising” showed a significant upward temporal trend (Mann-Kendall tau of 0.607, P< 0.001).

Fig. 8 Graph with percentages of articles with the positive word “robust” in the title/abstract between 2003 and 2022. The gray line denotes the proportions (%) per year and the blue line represents the non-parametric locally estimated scatterplot smoothing (LOESS) fit in each graph. Usage of “robust” showed a significant upward temporal trend (Mann-Kendall tau of 0.737, P< 0.001).

Fig. 9 Graph with percentages of articles with the positive word “unique” in the title/abstract between 2003 and 2022. The gray line denotes the proportions (%) per year and the blue line represents the non-parametric locally estimated scatterplot smoothing (LOESS) fit in each graph. Usage of “unique” showed a significant upward temporal trend (Mann-Kendall tau of 0.747, P< 0.001).

Association between positive word usage and journal IF

There was no significant association between positive word usage and journal IF (β coefficient of 0.006 per unit increase in journal IF, P = 0.917).

Post hoc analysis: temporal trends in the use of individual neutral words

Five neutral words showed significantly increased usage over time and were present in more than 1 % of titles/abstracts in at least one year: “brain” (Mann-Kendall tau of 0.747, P = 0.002), “disease” (Mann-Kendall tau of 0.385, P = 0.019), “intervention” (Mann-Kendall tau of 0.522, P = 0.001), “prospective” (Mann-Kendall tau of 0.702, P< 0.001), and “treatment” (Mann-Kendall tau of 0.617, P< 0.001).

Discussion

The results of this study show that the use of positive words in the titles/abstracts of publications in general radiology journals has strongly increased between 2003 and 2022. Negative word usage remained stable in the same time span. This suggests that researchers in the field of radiology are increasingly promoting their research findings. Increasing use of promotional language is thought to be a reaction of researchers to increasing publication pressure and publication bias favoring positive results over negative results [2] [3] [4] [5] [6]. Note that previous research has shown that radiology researchers also perceive publication bias as a widespread phenomenon [9]. Although the increasing use of promotional language may not directly equal scientific misconduct, it can be questioned if this practice represents another type of research “spin” [10]. Spin has been defined as specific intentional or unintentional reporting that fails to faithfully reflect the nature and range of findings and that could affect the impression the results produce in readers [10]. Readers of scientific articles may indeed be misled when irrelevant, doubtful, or still unproven research findings are misrepresented with positive wording. This, in turn, may potentially lead to the clinical implementation of research findings that are useless or even harmful, and the execution of additional, useless research on the same topic that may be a waste of resources. On the other hand, readers of general radiology journals may also lose their confidence in the reliability of the research that is published. This may potentially lead them to ignore important research findings that could actually be beneficial to clinical practice or that deserve further investigation. Altogether, the overuse of positive words may disrupt the perception and credibility of radiology research findings, and negatively affect clinical decision making and future research. The fact that the frequency of positive words in publications has increased over the past two decades also suggests that the use of positive words increases the chance of getting an article accepted for publication, basically amplifying the phenomenon of publication bias. Journal editors and reviewers may have to be more vigilant to check that the words that are used by scientists accurately reflect the nature of their findings. Special attention can be paid to the words “excellent”, “favorable”, “promising”, “robust”, and “unique”. Journals could establish author guidelines for balancing enthusiastic language with factual accuracy to maintain the integrity of research communication and instruct reviewers to ensure adherence to these guidelines. Because many authors also perform peer reviews for journals, such a policy may potentially reverse the overuse of positive words. The increase in positive word usage was found throughout the entire range of journal IFs, which indicates that addressing this topic is relevant to all journals. Interestingly, the use of neutral words also showed a significant temporal increase over time, albeit moderately. The neutral words “brain”, “disease”, “intervention”, “prospective”, and “treatment” contributed most to this growth. This may reflect the types of research that have increasingly been performed between 2003 and 2022 and perhaps also due to the fact that the overall length of the abstracts has somewhat increased in this time period.

One previous study by Vinkers et al. [7] performed a similar analysis in the general medical literature by analyzing all abstracts in PubMed between 1974 and 2014. The authors of that study reported the absolute frequency of positive words to have increased from 2.0 % (1974–1980) to 17.5 % (2014), corresponding to a relative increase of 775 % [7]. The positive word usage in the radiological literature showed a similar upward temporal trend in the past two decades, although less steep compared to the general medical literature in the past four decades [7]. Nevertheless, the use of positive words in the radiological literature has continued to increase since 2014. Vinkers et al. [7] reported that the positive words “robust”, “novel”, “innovative”, and “unprecedented” were most prevalent. The word “robust” was also popular in the radiological literature, but otherwise different positive words were preferred by radiology researchers. Vinkers et al. [7] also reported that the increase in positive word use over the last 10 years of their research period was significantly lower in the group of high IF journals than the frequency pattern of positive words across all journals. It was speculated that this difference could be the result of a more thorough and critical editorial and peer review process in high IF journals [7]. However, there was no correlation between positive word usage and journal IF in the radiological literature. It can be argued that the increase in positive words may be attributed to general language trends in society. However, a previous study by Vinkers et al. [7] already showed that this is likely not the case, at least for the period between 1974 and 2014. Whether or not the spread of social media has any association with positive word usage in scientific literature remains unclear. Further research is necessary to investigate the impact of positive language on readers' perceptions and citation rates of articles in radiology. Future research should also investigate if positive word usage will continue to increase in the coming years.

The present study had some limitations. First, limited sets of positive, negative, neutral, and random words were used. However, these sets of words had been validated by previous similar research published in the British Medical Journal that also showed that the addition of more positive words yielded similar results [7]. Second, because of the vast amount of data (94,550 articles were included) and search limitations in PubMed, it was only possible to search in titles and abstracts for the presence of any positive, negative, neutral, or random word, rather than analyzing the entire article text for the total number of words in each category. Third, individual (positive) words were not assessed in the context of their use. Therefore, the exact intent and weight of these words in the titles and abstracts of the articles that were analyzed, and whether or not they truly represent “overselling” of research findings remain unclear.

Conclusion

In conclusion, radiology researchers appear to increasingly promote their scientific findings by more frequently using positive words in their publications over the past two decades.

• References

• 1 Abbott A, Cyranoski D, Jones N. et al. Metrics: Do metrics matter?. Nature 2010; 465: 860-862
• 2 Tijdink JK, Vergouwen AC, Smulders YM. Publication pressure and burn out among Dutch medical professors: a nationwide survey. PLoS One 2013; 8 (09) e73381
• 3 Haven TL, Bouter LM, Smulders YM. et al. Perceived publication pressure in Amsterdam: survey of all disciplinary fields and academic ranks. PLoS One 2019; 14: e0217931
• 4 Suart C, Neuman K, Truant R. The impact of the COVID-19 pandemic on perceived publication pressure among academic researchers in Canada. PLoS One 2022; 17 (06) e0269743
• 5 Treanor L, Frank RA, Cherpak LA. et al. Publication bias in diagnostic imaging: conference abstracts with positive conclusions are more likely to be published. Eur Radiol 2020; 30 (05) 2964-2972
• 6 Treanor LM, Frank RA, Atyani A. et al. Reporting bias in imaging diagnostic test accuracy studies: are studies with positive conclusions or titles submitted and published faster?. Am J Roentgenol 2021; 216 (01) 225-232
• 7 Vinkers CH, Tijdink JK, Otte WM. Use of positive and negative words in scientific PubMed abstracts between 1974 and 2014: retrospective analysis. BMJ 2015; 351: h6467
• 8 Journal Citation Reports (InCites). 2021 https://jcr.clarivate.com/ Accessed 20 April 2023.
• 9 Kwee RM, Almaghrabi MT, Kwee TC. Scientific integrity and fraud in radiology research. Eur J Radiol 2022; 156: 110553
• 10 Boutron I, Ravaud P. Misrepresentation and distortion of research in biomedical literature. Proc Natl Acad Sci U S A 2018; 115: 2613-2619