J Pediatr Intensive Care 2021; 10(01): 083-084
DOI: 10.1055/s-0040-1713612
Letter to the Editor

Truth Has Nothing to Do with the Conclusion, and Everything to Do with the Methodology

Vicent Modesto i Alapont
1   Pediatric Intensive Care Unit, Department of Pediatrics, Hospital Universitari i Politècnic La Fe, Valencia, Spain
,
Alberto Medina
2   Pediatric Intensive Care Unit, Department of Pediatrics, Department of Pediatrics, Hospital Universitario Central de Asturias, Oviedo, Spain
,
3   Department of Pediatrics, Hospital Universitario Río Hortega, Valladolid, Spain
› Author Affiliations

High-Flow Nasal Cannula versus Continuous Positive Airway Pressure in Critical Bronchiolitis: A Randomized Controlled Pilot

Response from the Authors

We would like to begin this letter with a simple scientific statement: it is not possible to claim one thing and within the same logical argument claim the complete opposite. In the randomized controlled trial (RCT) of high flow nasal cannulae (HFNC) versus continuous positive airway pressure (CPAP) in critical bronchiolitis published in Journal of Pediatric Intensive Care, we have an example of this fallacy.[1] The authors explain the following:

  • Considering a 1:1 treatment allocation, a statistical signification level alfa (probability of committing type-I error) of 5%, two relative noninferiority margins of 15% (same as in the TRAMONTANE study[1]) and a more conservative 10%, and inflating the sample size estimate by 5% to account for the possibility of missing or unusable data in the final analysis.

  • Considering the finding of treatment failure in 10 out of 28 infants in the CPAP group (35.7%; 95% confidence interval: 18.0–53.5%).

  • To have the power of correctly confirming a true null hypothesis (H0) of 80% (probability of committing type-II error = 20%) in a RCT of HFNC versus CPAP, a total of 3,015 patients, or 1,583 patients per group (in the TRAMONTANE study assumption), either a total of 6,475 patients, or 3,400 patients in each group (if the more conservative assumption holds), is required.

With this technical jargon, the authors are saying that if the experiment is not able to detect a statistically significant difference in failure rates between two treatments, either the sample trial has a great number of patients or it is not methodologically correct to conclude that H0 is true. In other words, with only 63 patients, it is impossible to confirm the equivalence in treatment failures between HFNC and CPAP without a high risk of false-negative findings.

Having stated this, from a scientific point of view, the following sentences are not acceptable in the same text: “… comparing HFNC therapy and CPAP in 63 infants with critical bronchiolitis, we found that both modalities had similar rates of treatment failure” or “in this pilot study, treatment with HFNC resulted in a rate of treatment failure similar to CPAP.” The reason is that it is not possible to conclude contrary things in the same logical discussion.

With the same assumptions, in a RCT with a sample size of 63 patients (35 in the experimental HFNC arm), if the true failure rate for the experimental group is obtained from TRAMONTANA study, the power of the study to confirm H0 is only 0.149. With the most conservative study assumption the power would be 0.076 ([Fig. 1]).[3] [4] These are the probabilities provided by Cesar et al that the failure rates of HFNC and CPAP are equal (accepting H0). In fact, it is overwhelmingly probable that the set of alternative hypotheses of nonequivalence between HFNC and CPAP were true. So, “we were not able to refute the hypothesis of equivalence between HFNC and CPAP” would seem to be a more accurate logical conclusion.

Zoom Image
Fig. 1 Sample size computation: α (type-I error probability) = 0.05; β (type-II error probability) = 0.2; power of the study = 80%; treatment allocation = 28:35 (0.8:1); treatment failure in CPAP group = 0.357. Margins of noninferiority: TRAMONTANE study (red line) = 15% (treatment failure in HFNC group = 0.507); conservative assumption (blue line) = 10% (treatment failure in HFNC group = 0.457). Made with software “PS-Power and Sample size calculations”.[2] [3]

Regarding the statistical analysis, in a small-sample RCT, Bayesian analysis might be more accurate. Thus, a more adjusted-to-data conclusions could be obtained using Bayes rule in a beta-binomial model and a noninformative beta (1/2, 1/2) prior distribution. Using this analysis and data by Cesar et al, the probability that the intubation rate was higher in the HFNC group than in the CPAP group is 0.837.

In conclusion, in a scientific investigation, “truth has nothing to do with the conclusion, and everything to do with the methodology.”



Publication History

Received: 21 April 2020

Accepted: 14 May 2020

Article published online:
21 July 2020

© 2020. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 
  • References

  • 1 Milési C, Essouri S, Pouyau R. et al; Groupe Francophone de Réanimation et d’Urgences Pédiatriques (GFRUP). High flow nasal cannula (HFNC) versus nasal continuous positive airway pressure (nCPAP) for the initial respiratory management of acute viral bronchiolitis in young infants: a multicenter randomized controlled trial (TRAMONTANE study). Intensive Care Med 2017; 43 (02) 209-216
  • 2 Cesar RG, Bispo BRP, Félix P. et al. High-flow nasal cannula versus continuous positive airway pressure in critical bronchiolitis: a randomized controlled trial. J Pediatr Intensive Care 2020 DOI: 10.1055/s-0040-1709656
  • 3 Dupont WD, Plummer Jr WD. Power and sample size calculations. A review and computer program. Control Clin Trials 1990; 11 (02) 116-128
  • 4 Casagrande JT, Pike MC. An improved approximate formula for calculating sample sizes for comparing two binomial distributions. Biometrics 1978; 34 (03) 483-486