An Unconditional Test for Change Point Detection in Binary Sequences with Applications to Clinical Registries

Summary

Objectives: Methods for change point (also sometimes referred to as threshold or breakpoint) detection in binary sequences are not new and were introduced as early as 1955. Much of the research in this area has focussed on asymptotic and exact conditional methods. Here we develop an exact unconditional test.

Methods: An unconditional exact test is developed which assumes the total number of events as random instead of conditioning on the number of observed events. The new test is shown to be uniformly more powerful than Worsley’s exact conditional test and means for its efficient numerical calculations are given. Adaptions of methods by Berger and

Boos are made to deal with the issue that the unknown event probability imposes a nuisance parameter. The methods are compared in a Monte Carlo simulation study and applied to a cohort of patients undergoing traumatic orthopaedic surgery involving external fixators where a change in pin site infections is investigated.

Results: The unconditional test controls the type I error rate at the nominal level and is uniformly more powerful than (or to be more precise uniformly at least as powerful as) Worsley’s exact conditional test which is very conservative for small sample sizes. In the application a beneficial effect associated with the introduction of a new treatment procedure for pin site care could be revealed.

Conclusions: We consider the new test an effective and easy to use exact test which is recommended in small sample size change point problems in binary sequences.

Keywords

Change point - unconditional test - binary segmentation - logistic regression - observational study

• References

• 1 Page ES. A test for a change in a parameter occurring at an unknown point. Biometrika 1955; 42: 523-527.
• 2 Worsley KJ. The power of likelihood ratio and cumulative sum tests for a change in a binomial probability. Biometrika 1983; 70: 455-64.
• 3 Halpern A. Minimally Selected p and Other Tests for a Single Abrupt Changepoint in a Binary Sequence. Biometrics 1999; 55: 1044-1050.
• 4 Friede T, Henderson R, Kao CF. A note on testing for intervention effects on binary responses. Methods Inf Med 2006; 45 (Suppl. 04) 435-40.
• 5 Fong Y, Di C, Permar S. Change point testing in logistic regression models with interaction term. Stat Med 2015; 34: 1483-1494.
• 6 Dimick JB, Ryan AM. Methods for Evaluating Changes in Health Care Policy: The Difference-inDifferences Approach. JAMA 2014; 312 (Suppl. 22) 2401.
• 7 Worsley KJ. Confidence regions and tests for a change-point in a sequence of exponential family variables. Biometrika 1986; 73: 91-104.
• 8 Assareh H, Smith I, Mengersen K. Change point detection in risk adjusted control charts. Stat Methods Med Res 2015; 24 (Suppl. 06) 747-68.
• 9 Barnard GA. A new test for 2×2 tables. Nature 1945; 156 177: 783-784.
• 10 Boschloo RD. Raised conditional level of significance for the 2×2 table when testing the equality of probabilities. Statistica Neerlandica 1970; 24: 1-35.
• 11 Mehrotra DV, Chan ISF, Berger RL. A Cautionary Note on Exact Unconditional Inference for a Difference between Two Independent Binomial Proportions. Biometrics 2003; 59 (Suppl. 02) 441-50.
• 12 Chen J, Gupta AK. Parametric statistical change point analysis: with applications to genetics, medicine, and finance. 2nd ed. New York: Birkhäuser; 2012
• 13 Kaptchuk TJ. Commentary – The double-blind, randomized, placebo-controlled trial: Gold standard or golden calf?. J Clin Epidemiol 2001; 54: 541-549.
• 14 Heuer C, Abel U. The analysis of intervention effects using observational data bases. In Nonrandomized comparative clinical studies. Abel U, Koch A. (eds). Symposium Publishing; Düsseldorf: 1998: 101-107.
• 15 Friede T, Henderson R. Intervention effects in observational survival studies with an application in total hip replacements. Stat Med 2003; 22: 3725-3737.
• 16 Pater J, Rochon J, Parmar M, Selby P. Future research and methodological approaches. Ann Oncol 2001; 22 (Suppl. 07) vii57-vii61.
• 17 Davies R, Holt N, Nayagam S. The care of pin sites with external fixation. J Bone Joint Surg Br 2005; 87 (Suppl. 05) 716-9.
• 18 Suissa S, Shuster JJ. Exact Unconditional Sample Sizes for the 2 x 2 Binomial Trial. J R Statist. Soc. A 1985; 148 Part 4 317-327.
• 19 Berger RL, Boos D. P-values maximized over a confidence set for the nuisance parameter. J Am Stat Assoc 1994; 89: 1012-1016.
• 20 Straube S, Werny B, Friede T. A systematic review identifies shortcomings in the reporting of crossover trials in chronic painful conditions. J Clin Epidemiol 2015; 68 (Suppl. 12) 1496-1503.
• 21 von Elm E, Altmanc DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for reporting observational studies. Prev Med 2007; 45 (Suppl. 04) 247-251.
• 22 Halpern AL. Multiple-changepoint testing for an alternating segments model of a binary sequence. Bio-metrics 2000; 56: 903-908.