Keywords
comparative effectiveness - explanatory trial - individualised homeopathic treatment
- meta-analysis - pragmatic trial - randomised controlled trial - risk of bias - systematic
review
Introduction
There are several distinct forms of homeopathy, the main types being ‘individualised’
(or ‘classical’) homeopathy, ‘clinical’ homeopathy, ‘complex’ homeopathy and isopathy.
In individualised homeopathy—as originally defined by its founder, Samuel Hahnemann—typically a single homeopathic
medicine is selected on the basis of the ‘symptom picture’ of a patient, including
his/her mental, general and constitutional type. In clinical homeopathy, one or more homeopathic medicines are administered for standard clinical situations
or conventional diagnoses. In complex homeopathy, several homeopathic medicines are combined in a fixed (‘complex’) formulation. Isopathy is the use of homeopathic medicines prepared from the causative agent of the disease
itself, or from a product of the disease process, to treat the condition.[1]
Our previous reviews focused on individualised or non-individualised homeopathy in the context of placebo-controlled randomised controlled trials (RCTs).[2]
[3] The current study focuses again on the ‘whole-person’ therapeutic approach of individualised homeopathic treatment (IHT): individualised consultation together with the consequently prescribed homeopathic medicine/s (and sometimes including dietary
and/or lifestyle recommendations). The context here is an RCT in which the control
(comparator) group is something other than placebo (OTP), and which can be regarded as a ‘comparative effectiveness’ study. Whereas
for placebo-controlled trials of IHT, we investigated whether individually prescribed
homeopathic medicines can have effects above those of placebos,[2] our study of OTP-controlled trials aims to ascertain the comparative effectiveness
of IHT as a whole entity: that is, the package of homeopathic care that includes the prescribed medicines.
Two essentially different comparator options exist for OTP study design of RCTs: (I)
other therapeutic intervention (e.g. a conventional medicine or a physical therapy),
which can be sub-divided into (a) trials in which IHT is given as an alternative to the comparator intervention, and (b) trials in which IHT combined with the other intervention is compared with the other intervention alone (the ‘[A + B] versus B’ approach);
(II) no therapeutic intervention (usually waiting-list controls).
Some systematic reviews (SRs) of homeopathy have focused on specific clinical conditions,
typically examining placebo-controlled trials, with conclusions about efficacy of
homeopathic medicines that are variously positive,[4]
[5]
[6] negative[7]
[8]
[9] or non-conclusive.[10]
[11]
[12] Other SRs have adopted a ‘global’ or ‘comprehensive’ approach, examining the RCT
research literature on homeopathy in general, including the broad spectrum of clinical
conditions that have been researched, and by all forms of homeopathy. Most of these
comprehensive SRs reached the cautious conclusion that, overall, the effect of a homeopathic
medicine differs from that of placebo.[2]
[13]
[14]
[15]
[16]
[17] One comprehensive SR concluded that there was ‘weak evidence for a specific effect
of homoeopathic remedies…compatible with the notion that the clinical effects of homoeopathy
are placebo effects’.[18] There has been insufficient quantity or quality of evidence in these comprehensive
SRs—including a recent SR of placebo-controlled RCTs in non-individualised homeopathic
treatment[3]—to enable clear conclusions about the efficacy of homeopathic medicines for any
specific clinical condition.
Only one general SR has considered solely RCTs of IHT that were controlled by an OTP
intervention.[19] Published 19 years ago, this SR identified just six eligible trials, each of which
was judged to contain serious methodological flaws. Findings for these trials were
mixed: two favoured IHT, two favoured conventional drugs and two were non-conclusive.
Each trial investigated a different clinical condition, and none had been replicated
by an independent research group. It was concluded that the ‘value of individualised
homoeopathy relative to allopathic treatments’ was not known.
An RCT's lack of bias (its internal validity), the generalisability of its findings
(external validity) and the quality of its clinical intervention (model validity)
are key to its overall worth. Though, in general, it is seldom formally addressed
by trialists or in SRs,[20]
[21] external validity can be appraised using a tool (PRECIS), developed by Thorpe et al in 2009, that considers a trial design's positioning
on ‘the pragmatic–explanatory continuum’.[22] For the current SR of OTP-controlled trials, we assess each study's internal validity
as well as its external validity, with a view to appraising both its intrinsic quality
and the extent of its pragmatic/explanatory attitude.
Aim of the Study
Our objective was to examine comparative effectiveness of IHT in OTP-controlled trials
of any clinical condition, in adults or children, for which there was at least one
eligible RCT. Using meta-analysis methods, we aimed to evaluate RCTs that have investigated
IHT: (study design Ia) in comparison to another therapeutic intervention; (study design
Ib) adjunctively with another treatment intervention in comparison to the other intervention
alone (‘[A + B] versus B’); or (study design II) compared with no other intervention.
An additional aim was, where possible, to evaluate by meta-analysis the comparative
effectiveness of IHT for any given clinical condition or category of conditions. In
all cases, we reflected matters of internal validity (risk of bias) and external validity
(pragmatic/explanatory study attitude).
Methods
Methods comply fully with the PRISMA 2009 Checklist ([Supplementary File 1], available in online version only) and with our published study protocol[23] (not registered with PROSPERO).
Search Strategy, Data Sources and Trial Eligibility
We conducted a systematic literature search to identify RCTs that compared IHT with
something other than placebo, for any clinical condition. Each of the following electronic
databases had previously been searched from its inception up to the end of 2011: AMED;
CAM-Quest; CINAHL; Cochrane Central Register of Controlled Trials; Embase; Hom-Inform;
LILACS; PubMed; Science Citation Index; Scopus.[24] For the present SR, the same database searches were updated—each with the same strategy
as for 2011—to the end of 2015. For the update, CORE-Hom was also searched, using
the term ‘randomized’ or ‘unknown’ in the Sequence Generation field.
The full electronic search strategy for PubMed (Cochrane Highly Sensitive Search Strategy)
is given in our previous article:[24] ‘(homeopath* or homoeopath*) and (randomised controlled trial[pt]) or (controlled clinical trial[pt]) or (randomised [tiab]) or (placebo [tiab]) or (clinical trials as topic[mesh:noexp]) or (randomly [tiab]) or (trial[ti]) not (animals[mh] not humans[mh])’.
Specific pre-defined exclusion criteria were then applied:
-
Trials of homeopathic prophylaxis
-
Trials with crossover design
-
Research using radionically prepared ‘homeopathic’ medicines[25]
-
The tested intervention is IHT in conjunction with other (complementary or conventional)
medicine or therapy, and where the nature of the combined comparator intervention
makes it impossible to distinguish any effects due to IHT.[a]
-
Other specified reason.
Whereas a placebo-controlled trial of individualised homeopathy can be fully blinded,
it is more difficult—and sometimes impossible—to achieve such blinding in a corresponding
OTP-controlled trial. Unlike the case for our SR of placebo-controlled trials, therefore,
patient- and practitioner-unblinded trials are eligible for the current SR of OTP trials.
Only published data were eligible for analysis. Because it is recognised that contacting
the original authors of RCTs may lead to limited or overly positive answers,[26] the authors of eligible articles were not approached for clarification on unclear
or missing facts; however, original authors' cross-reference to their previously published
study methods were followed up and taken into account as necessary. For trials with
more than two study groups, and where such trials had not previously been catalogued
under ‘placebo-controlled’, only the data concerning comparisons between IHT and OTP
were extracted from the articles; in relevant cases of more than one OTP control,
a study group comprising actual treatment was favoured for analysis over one comprising
‘no treatment’.
Outcome Definitions
For each trial, and for the purposes of risk-of-bias assessment, we identified a ‘main
outcome measure’ using a refinement of the approaches adopted by Linde et al and by
Shang et al.[15]
[18] As for our previous reviews,[2]
[3] each trial's ‘main outcome measure’ was identified based on the following hierarchical
ranking order (consistent with the WHO ICF Classification System for Levels of Functioning
Linked to Health Condition):[b]
-
Mortality
-
Morbidity
-
Health impairment (loss/abnormality of function, including presence of pain)
-
Limitation of activity (disability, including days off work/school because of ill
health)
-
Restriction of participation (quality of life)
-
Surrogate outcome (e.g. blood test data, bone mineral density).
We followed the WHO ICF system without reference to any ‘primary outcome measure’
that may have been identified by the original investigators. Unless otherwise indicated,
the single end-point (measured from the start of the intervention) associated with
our designated ‘main outcome measure’ was taken as the last follow-up at which data
were reported for that outcome.
Data Extraction
Two reviewers (RTM and SU-Z) independently extracted relevant data using a standard
data recording approach, in spreadsheet format (Microsoft Excel). The data extracted per trial included, as appropriate: demographics of participants
(gender, age range, clinical condition); study setting; potency or potencies of homeopathic
medicines; dosage frequency; whether a pilot trial; ‘main outcome measure’ and measured
end-point; other outcome measures reported; funding source/s. The statistical items
noted were sample size and missing data for each intervention group; whether power
calculation performed; whether intention-to-treat (ITT), per-protocol, complier-average-causal-effect[27] or other type of primary analysis.
Assessment of Risk of Bias (Internal Validity)
Using the standard criteria defined by Cochrane,[26] the extraction of information enabled appraisal of ‘low risk’, ‘uncertain risk’
or ‘high risk’ of bias with respect to: (domain I) the methods used to generate the
random sequence; (domain II) the method of allocation concealment used to implement
the random sequence; (domain IIIA) the blinding of participants and/or study personnel;
(domain IIIB) the blinding of outcome assessors; (domain IV) completeness of the outcome
data included in the article's analysis; (domain V) evidence of selective outcome
reporting; (domain VI) evidence of other bias, including data imbalance between the
groups at baseline.
Two assessors (RTM and SU-Z) performed their assessments independently, with discrepancies
between them resolved by consensus discussion and, if necessary, the input of a third
assessor (LAL or JRTD). For domain IV, a trial was normally regarded as no better
than ‘unclear’ if there was greater than 20% participant attrition rate, irrespective
of whether ITT analysis had been performed. For domain V, judgment on selectiveness
of Results reporting in a given article was based on a comparison not with an original
trial protocol (for few exist in the public domain for RCTs of homeopathy), but rather
with details for outcomes provided in that same article's Methods section. Also, domain
V was automatically designated ‘high risk of bias’ if its main outcome measure could
not be extracted to enable calculation of ‘relative effect size’ (see below). The
source of any research sponsorship was taken into account for sub-group analysis (see
below), not in risk-of-bias assessment per se.
Rating of Trials for Risk of Bias
As per the standard Cochrane approach, each trial was designated: low risk of bias for all key domains; or uncertain risk of bias for one or more key domains; or high risk of bias for one or more key domains.[26] We used our novel method of nomenclature, based on the Cochrane approach, for rating
risk-of-bias characteristics across all domains per trial:[2]
[3]
-
A = Low risk of bias in all seven domains.
-
Bx = Uncertain risk of bias in x domains; low risk of bias in all other domains.
-
Cy.x = High risk of bias in y domains; uncertain risk of bias in x domains; low risk of bias in all other domains.
An ‘A’-rated trial is designated reliable evidence. We also designate a ‘B1’-rated trial reliable evidence (and listed as ‘B1* [minimal risk of bias]’) if the uncertainty in its risk of bias
is for one of domains IV, V or VI only (i.e. it is required to be judged free of bias
for each of domains I, II, IIIA and IIIB).[23]
It is expected that an OTP-controlled trial will be rated ‘high risk of bias’ in assessment
domain IIIA. We recognise that this is a normal feature of an OTP-controlled trial,
and which thus inevitably limits its internal validity—see also the section Sensitivity analyses, below.
Assessment of Pragmatic/Explanatory Attitude (External Validity)
Equating external validity to study attitude, we adopted the PRECIS approach[22] to assess each trial's positioning on the pragmatic–explanatory continuum, taking
account of the 10 domains:
-
Participant eligibility criteria;
-
Experimental intervention flexibility;
-
Experimental intervention practitioner expertise;
-
Comparison intervention;
-
Comparison intervention practitioner expertise;
-
Follow-up intensity;
-
Primary trial outcome;
-
Participant compliance with ‘prescribed’ intervention;
-
Practitioner adherence to study protocol;
-
Analysis of primary (‘main’) outcome.
Against a set of standard judgmental criteria,[22] two of us (RTM and PV) independently assessed each of the 10 attributes as ‘much
more explanatory than pragmatic’, ‘more explanatory than pragmatic’, ‘equally pragmatic
and explanatory’, ‘more pragmatic than explanatory’ or ‘much more pragmatic than explanatory’.
Consistent with the PRECIS authors' receptiveness to methodological innovation,[22] we devised our own notation, giving a score of 1, 2, 3, 4 or 5 in each case respectively.
Where an article contained no relevant information to enable assessment of a given
domain, a score of 3 was given. The range of possible total scores per trial was 10
to 50. The full PRECIS-based approach that we used to assess each domain is given in [Supplementary File 3] (available in online version only).
We characterised each trial overall using the following empirical consideration of
its total score:
-
10–17: Much more explanatory than pragmatic;
-
18–25: More explanatory than pragmatic;
-
26–34: Equally pragmatic and explanatory;
-
35–42: More pragmatic than explanatory;
-
43–50: Much more pragmatic than explanatory.
Discrepancies between assessments (per attribute and by overall characterisation per
trial) were resolved by consensus discussion. Initial attention focused on each domain
for which the two independent scores were a value of 2 or more apart, and with the
aim of narrowing the discrepancy to no more than a value of 1. For any studies whose
two sets of total scores lay on either side of a threshold for a given pragmatic/explanatory
attitude, we agreed a final designation by overall re-appraisal and discussion; we
did not calculate the mean value of the two independent total PRECIS scores.
Study Selection for Meta-Analysis
All RCTs that were included in the SR were potentially eligible for meta-analysis.
If the original article did not provide or inform adequate data on the selected ‘main
outcome measure’ to enable extraction or calculation of the SMD or the OR, we described
the selected main outcome as ‘not estimable’: an alternative, estimable, outcome was
not sought.
Consistent with the above, the following studies were excluded from meta-analysis:
-
Those that presented non-parametric data only, and where there was no information
that enabled the data distribution to be assessed;
-
Those from which the necessary data could not be extracted (not provided or uninterpretable).
Summary Measures for ‘Main Outcome’
For the remaining relevant records of IHT, we aimed to examine: (1) overall relative
effect sizes; (2) disease-specific relative effect sizes; (3) disease category-specific
relative effect sizes. In each of these three cases, ‘relative effect size’ was taken
as the difference (if relevant—see below) between the homeopathy and the control groups
at the identified end-point of the trial, and using per-protocol data:
-
For dichotomous measures: odds ratio (OR), with 95% confidence interval (CI);[c]
-
For continuous measures: standardised mean difference (SMD), with 95% CI.[d]
In trials where the main outcome measure was a continuous variable, and where there
were insufficient data presented to identify the mean and/or the standard deviation
(SD) per group at the defined end-point, the necessary data were estimated, if possible,
by imputing relevant other data from the same study.[28]
Statistical Interpretation Per Study Design
Interpretation of statistical finding: p ≤ 0.05 (direction of effect towards homeopathy):
-
(Ia) Other-intervention control: IHT is more effective than the other intervention;
-
(Ib) ‘[A + B] versus B’: IHT+ other intervention is more effective than the other intervention alone;
-
(II) No-treatment control: IHT is more effective than no intervention.
Interpretation of statistical finding: p ≤ 0.05 (direction of effect towards control):
-
(Ia) Other-intervention control: IHT is less effective than the other intervention;
-
(Ib) ‘[A + B] versus B’: IHT+ other intervention is less effective than the other intervention alone;
-
(II) No-treatment control: IHT is ineffective.
Interpretation of statistical finding: p > 0.05 (direction of effect towards either homeopathy or control):
-
(Ia) Other-intervention control: Inconclusive whether IHT and the other intervention differ in effectiveness;
-
(Ib) ‘[A + B] versus B’: Inconclusive whether the effectiveness of IHT+ other intervention differs from that
of the other intervention alone;
-
(II) No-treatment control: IHT is probably ineffective.
Detailed interpretation of the above is described in the study protocol[23] and reflects our approach to superiority trials. We separately interpreted studies explicitly designed as ‘equivalence’ or
‘non-inferiority’ trials,[29]
[30] reflecting the original authors' margin of equivalence or non-inferiority as appropriate.
For any RCT or group of RCTs on a given clinical condition/category, the interpretation
of IHT as ‘effective’, ‘ineffective’ or ‘inconclusive’ applied solely to the particular
clinical condition/category examined.
Synthesis of Quantitative Results
1) Overall ‘relative effect size’ of IHT
For groups of eligible RCTs that have compared IHT (Ia) with another intervention,
or (Ib) adjunctively with another intervention, or (II) with no treatment, we aimed
to pool for meta-analysis the ‘main outcome’ data in two separate sets of studies
as appropriate, using (1) the OR or (2) the SMD of each relevant trial.[31] For each study design (Ia, Ib, II), it was then planned to combine data from the
two sets of studies (OR and SMD) into a single forest plot, re-expressing SMDs by
transformation to OR, using an approximation method proposed by Chinn[32] and recommended by the Cochrane Statistical Methods Group.[33]
Based on the assumption of at least moderate clinical heterogeneity, the ‘random effects’
statistical model for meta-analysis was selected rather than the ‘fixed effect’ model.[31]
2) Disease- and category-specific ‘relative effect size’ of IHT
For each specific clinical condition or category of conditions, for each of study
designs (Ia), (Ib) and (II), and for which there was >1 RCT of given type and with
extractable main outcome, we planned to pool the data using meta-analysis.
3) Heterogeneity and asymmetry
The I
2 statistic was used to assess the variability between studies in a given meta-analysis:
it gives the percentage of the total variability in the estimated effect size (which
is composed of between-study heterogeneity plus sampling variability) that is attributable
to heterogeneity. The I
2 statistic can take values between 0 and 100%: I
2 = 0% means that all of the heterogeneity is due to sampling error; I
2 = 100% means that all variability is due to true heterogeneity between studies. Where
feasible, it was intended to use funnel plots to assess the impact of publication
bias.
Additional Quantitative Analyses on Overall ‘Relative Effect Size’ of IHT
Sensitivity Analyses
We planned sensitivity analyses based separately on (1) our risk-of-bias ratings and
on (2) our assessments of external validity. We aimed to reflect in sensitivity analysis
any trial that was categorised as ‘(much) more pragmatic than explanatory’ and whose
internal validity was limited by high risk of bias in domain IIIA only (blinding of participants and/or study personnel)—see also Rating of trials for risk of bias, above.
Sub-group Analyses
Comparative forest plots were planned—for each of study designs (Ia), (Ib) and (II)—on
the following sub-groups of trial attributes:
-
Whether or not a pilot (or ‘preliminary’ or ‘feasibility’) study, as defined by the
original authors;
-
Whether or not sample size > median for all trials with extractable data;
-
Whether or not potency/potencies of homeopathic medicines ≥ 12C;
-
Whether or not the research sponsor is an organisation (e.g. homeopathic pharmacy)
with potential vested interest in the trial findings.
Results
Included Studies
The PRISMA flowchart from the original comprehensive literature search (up to and including
2011) was published previously.[24] An updated PRISMA flowchart is given in [Fig. 1], identifying a total of 565 records;[e] 466 remained after removal of duplicates. After excluding 97 due to type of record
(book chapter, thesis, abstract and other minor article), 369 full-text records were
then assessed for eligibility. Three hundred and fifty-eight records were excluded
for the general reasons summarised in [Fig. 1]; seven of these same 358 were excluded from the present SR for the additionally
specified reasons shown in [Supplementary File 4] (available in online version only).[f] The finally remaining 11 records (11 RCTs) were thus included in this SR; data were
not extractable from three of those, leaving eight records potentially available for
meta-analysis—see [Supplementary File 4] for details of the three records excluded from quantitative analysis.
Fig. 1 Updated PRISMA flowchart for all records published up to and including 2015.
Characteristics of Included Studies
The 11 RCTs (publication details given in [Supplementary File 4]) represent 11 clinical conditions across eight categories ([Table 1]). Homeopathic potency was >12C in all five trials for which relevant information
was available. Seven trials were free of vested interest; one trial was not free of
vested interest; three trials did not enable certainty in this assessment.
Table 1
Demographic data for 11 OTP-controlled RCTs of individualised homeopathic treatment:
three studies excluded from meta-analysis are shown in italics
|
No.
|
First author
|
Year
|
Study design
|
Category
|
Condition
|
Participants' demographics
|
Study setting
|
Homeopathic potency; frequency
|
Comparator
|
Funding source
|
Free of vested interest [a]
|
|
A138
|
Adler
|
2009
|
Ia
|
Mental disorder
|
Depression
|
Referred patients who met DSM-IV criteria for depression
|
Outpatient Clinic of Homeopathy and Depression, Medical School, São Paulo, Brazil
|
Q; 3 times per week
|
[b]Fluoxetine; 20 mg once per day
|
HN-Pharmacy donated the homeopathic potencies
|
N
|
|
A140
|
Gmünder
|
2002
|
Ia
|
Musculoskeletal
|
Low back pain
|
Patients with chronic back pain lasting longer than 7 weeks: male and female patients,
aged 18–70 years
|
Department for Physical Medicine and Rheumatology, Orthopaedic University Clinic,
Switzerland
|
Not stated
|
[b]'Standard physiotherapy'; 8 sessions with stabilising, strengthening measures, supported
by heat applications and massage
|
None stated
|
U
|
|
A141
|
Harrison
|
1999
|
Ia
|
Ear, nose and throat
|
Otitis media (chronic)
|
Patients aged 18 months to 8 years with positive diagnosis of OME; hearing loss >20dB;
abnormal tympanogram
|
Two paediatric/ENT specialist centres in England
|
Not stated
|
Standard care ('watch and wait'); course of low-dose antibiotics for 4–6 weeks in
some cases
|
Research Council for Complementary Medicine
|
Y
|
|
A143
|
Paterson
|
2003
|
Ib
|
Gastroenterology
|
Dyspepsia
|
Patients with dyspepsia presenting in one general practice
|
Regional Medical Center, England
|
Not stated
|
Normal GP care
|
NHS South & West Region R&D project grant. Medical Centre funded as research general
practice
|
Y
|
|
A144
|
Relton
|
2009
|
Ib
|
Rheumatology
|
Fibromyalgia
|
Adults with formal diagnosis of primary fibromyalgia
|
Regional Rheumatology Department, England
|
Not stated
|
[b]Usual care: one or more of physiotherapy, aerobic exercise, analgesics, NSAIDs, anti-depressants
|
NHS Hospital Foundation Trust; Homeopathy Action Trust Charity; Department of Health
funded Centre for Research Capacity Development
|
Y
|
|
A148
|
van Erp
|
1996
|
Ia
|
Tropical disease
|
Malaria
|
Patients with standardised clinical signs of malaria, aged > 10 years
|
Clinic in Rural Ghana
|
200C; one granule
|
[b]Chloroquine; 25mg/kg body weight
|
None stated
|
U
|
|
A149
|
Witt
|
2009
|
Ia
|
Obstetrics and gynaecology
|
Vulvovaginal candidiasis
|
Women ≥18 years, with ≥ four episodes of Candida vaginitis in previous year and with
current acute symptoms
|
Medical School Outpatient Department, Austria
|
30–1,000C; frequency not stated
|
[b]Itraconazole; 200 mg b.i.d., 1 day per month, with and without local lactobacilli
|
‘None’
|
Y
|
|
A282
|
Relton
|
2012
|
Ia
|
Obstetrics and gynaecology
|
Menopausal syndrome
|
Women aged 45–64 years, experiencing 14 or more hot flushes or night sweats per week
|
Menopause Clinic in Sheffield, England
|
Not stated
|
Standard care: not specified (not HRT)
|
Department of Health National Co-ordinating Centre for Research Capacity Development
|
Y
|
|
A283
|
Sinha
|
2012
|
Ia
|
Ear, nose and throat
|
Otitis media (acute)
|
Children of either gender, aged 2–6 years; earache of not more than 36 hours duration;
bulging tympanic membrane
|
Regional Research Institute of Homeopathy, Jaipur, India
|
LM; 2–6 hourly, depending on need
|
[b]Symptomatic relief: analgesic, anti-inflammatory and antipyretic (antibiotics prescribed
if less than 50% improved at day 3)
|
None stated
|
U
|
|
A296
|
Peckham
|
2014
|
Ib
|
Gastroenterology
|
Irritable bowel syndrome
|
IBS diagnosis using Rome III criteria; aged ≥ 18
|
Regional Hospital Clinic, England
|
Not stated
|
Usual care: not specified
|
Barnsley Hospital Small Grants Fund; Friends of Barnsley Hospital; Homeopathy Research
Institute
|
Y
|
|
A304
|
Frass
|
2015
|
Ib
|
Miscellaneous
|
Cancer
|
All cancer patients aged > 18 years, prior to first conventional treatment (chemotherapy,
radiation therapy, surgical therapy)
|
Medical School Oncology Department, Vienna, Austria
|
Q, C, LM; daily
|
Conventional cancer therapy: chemotherapy, radiation, or surgery
|
‘None’
|
Y
|
Abbreviations: DSM-IV, Diagnostic and Statistical Manual of Mental Disorders, Fourth
Edition; HRT, hormone replacement therapy; NSAIDs, non-steroidal anti-inflammatory
drugs; OME, otitis media with effusion; OTP, other than placebo; RCT, randomised controlled
trial; N, no; U, unclear; Y, yes.
a Vested interest: Support (direct, through research sponsorship; indirect, via gifted
medicines) from company that provided homeopathic medicines for the trial.
b Best effective standard care.
Summary of Findings
For each trial, [Table 2] includes details of the sample size, the identified main outcome measure (and whether
dichotomous or continuous), the end-point, and several other study attributes. Six
trials were described in the original article as a ‘pilot’ (or ‘preliminary’ or ‘feasibility’)
study. A power calculation was performed for four of the trials. ITT was the basis
for original analysis in six trials. Mean attrition rate per trial was 20.1%. The
main outcome variable was dichotomous in four studies and continuous in the other
seven. The total sample size for the eight meta-analysable trials was 675; the median sample size was 59 (inter-quartile range, 45.5–75.5). The eight analysable
studies included eight different main outcome measures and for an end-point that ranged
from 7 days to 12 months.
Table 2
Summary of findings table: three studies excluded from meta-analysis shown in italics
|
Study design Ia: IHT versus other intervention
|
|
No.
|
First author
|
Year
|
Pilot
|
Power calc.
|
ITT sample
|
PP sample
|
PP sample > median (59)
|
Attrition rate %
|
Original primary analysis
|
‘Main’ outcome identified
|
Nature of 'main' outcome
|
End-point
|
|
A138
|
[a]Adler
|
2009
|
N
|
N
|
91
|
55
|
N
|
39.6
|
PP
|
MADRS score
|
Continuous
|
8 weeks
|
|
A140
|
Gmünder
|
2002
|
Y
|
N
|
43
|
37
|
|
14.0
|
PP
|
Oswestry Questionnaire Score
|
Continuous
|
8 weeks
|
|
A141
|
Harrison
|
1999
|
Y
|
N
|
35
|
33
|
N
|
5.7
|
PP
|
Treatment success: hearing loss of less than 20 dB
|
Dichotomous
|
12 months
|
|
A148
|
van Erp
|
1996
|
N
|
N
|
74
|
55
|
N
|
25.7
|
PP
|
Improvement in at least three of listed symptoms of malaria
|
Dichotomous
|
1 week (for daily attacks); 3 weeks (for weekly relapses)
|
|
A149
|
Witt
|
2009
|
N
|
Y
|
150
|
71
|
Y
|
52.7
|
PP
|
Candida-free status
|
Dichotomous
|
12 months
|
|
A282
|
Relton
|
2012
|
Y
|
N
|
41
|
40
|
|
2.4
|
ITT (included pts who refused Rx)
|
Hot flush frequency severity score
|
Continuous
|
36 weeks
|
|
A283
|
Sinha
|
2012
|
Y
|
N
|
81
|
80
|
Y
|
1.2
|
ITT (LOCF; but N/A at 7d end-point)
|
‘Cure rate’
|
Dichotomous
|
7 days duration
|
|
Study design Ib: IHT + other intervention versus other intervention alone
|
|
No.
|
First author
|
Year
|
Pilot
|
Power calc.
|
ITT sample
|
PP sample
|
PP sample > median (62.5)
|
Attrition rate %
|
Original primary analysis
|
‘Main’ outcome identified
|
Nature of 'main' outcome
|
End-point
|
|
A143
|
Paterson
|
2003
|
Y
|
N
|
40
|
32
|
|
20.0
|
ITT
|
MYMOP
|
Continuous
|
6 months
|
|
A144
|
Relton
|
2009
|
Y
|
Y
|
47
|
36
|
N
|
23.4
|
ITT/PP
|
FIQ total score
|
Continuous
|
22 weeks
|
|
A296
|
Peckham
|
2014
|
N
|
Y
|
72
|
63
|
Y
|
12.5
|
ITT/PP
|
Number who achieved clinically relevant change of 50% in IBS SSS
|
Continuous
|
26 weeks
|
|
A304
|
Frass
|
2015
|
N
|
Y
|
373
|
282
|
Y
|
24.4
|
ITT
|
Global health status
|
Continuous
|
4 months
|
Abbreviations: FIQ, fibromyalgia impact questionnaire; IBS, Irritable Bowel Syndrome;
IHT, individualised homeopathic treatment; ITT, intention to treat; MADRS, Montgomery
& Åsberg Depression Rating Scale; MYMOP, Measure Yourself Medical Outcome Profile;
PP, per protocol; SSS, Symptom Severity Score; N, no; Y, yes.
a Non-inferiority trial.
Risk of Bias
[Table 3] provides the risk-of-bias details for each of the 11 trials, and sub-divided by
study design: (Ia) the seven with other-intervention control; (Ib) the four that were
‘[A + B] versus B’. Domain IIIA (blinding of participants and/or study personnel),
domain IIIB (blinding of outcome assessors) and domain IV (completeness of outcome
data) presented the greatest methodological concerns. The three trials with high risk of bias for domain V were so because they did not contain data that were extractable for
meta-analysis. Domain I (sequence generation) and domain II (allocation concealment)
presented the most uncertain methodological judgments.
Table 3
Risk-of-bias and external validity assessments for RCTs: three studies excluded from
meta-analysis are shown in italics
|
Study design Ia: IHT versus other intervention
|
|
|
|
Risk-of-bias domain
|
|
|
|
|
No.
|
First author
|
Year
|
I
|
II
|
IIIA
|
IIIB
|
IV
|
[a]
V
|
VI
|
Risk of bias
|
Risk-of-bias rating
|
PRECIS assessment (total score per assessor)
|
|
A138
|
Adler
|
2009
|
Y
|
Y
|
Y
|
Y
|
N
|
Y
|
Y
|
High
|
C1.0
|
More explanatory than pragmatic (24; 24)
|
|
A140
|
Gmünder
|
2002
|
U
|
U
|
N
|
N
|
Y
|
[b]N
|
U
|
High
|
C3.3
|
Equally pragmatic and explanatory (34; 31)
|
|
A141
|
Harrison
|
1999
|
U
|
N
|
N
|
U
|
Y
|
Y
|
N
|
High
|
C3.2
|
Equally pragmatic and explanatory (33; 28)
|
|
A148
|
van Erp
|
1996
|
U
|
U
|
Y
|
U
|
N
|
Y
|
U
|
High
|
C1.4
|
Equally pragmatic and explanatory (33; 33)
|
|
A149
|
Witt
|
2009
|
Y
|
U
|
N
|
N
|
N
|
U
|
U
|
High
|
C3.3
|
More explanatory than pragmatic (21; 19)
|
|
A282
|
Relton
|
2012
|
U
|
Y
|
[c]N
|
N
|
Y
|
[b]N
|
N
|
High
|
C4.1
|
Much more pragmatic than explanatory (44; 43)
|
|
A283
|
Sinha
|
2012
|
Y
|
U
|
U
|
Y
|
Y
|
Y
|
Y
|
Uncertain
|
B2
|
Equally pragmatic and explanatory (31; 32)
|
|
Study design Ib: IHT + other intervention versus other intervention alone
|
|
A143
|
Paterson
|
2003
|
U
|
Y
|
[c]N
|
N
|
U
|
[b]N
|
Y
|
High
|
C3.1
|
More pragmatic than explanatory (38; 38)
|
|
A144
|
Relton
|
2009
|
Y
|
Y
|
[c]N
|
N
|
N
|
Y
|
Y
|
High
|
C3.0
|
More pragmatic than explanatory (36; 36)
|
|
A296
|
Peckham
|
2014
|
Y
|
Y
|
[c]N
|
Y
|
U
|
Y
|
Y
|
High[d]
|
C1.1
|
Much more pragmatic than explanatory (43; 45)
|
|
A304
|
Frass
|
2015
|
Y
|
Y
|
[c]N
|
N
|
N
|
Y
|
U
|
High
|
C3.1
|
More pragmatic than explanatory (37; 38)
|
Abbreviations: IHT, individualised homeopathic treatment; RCT, randomised controlled
trial; N, no (not free of bias); U, unclear; Y, yes (free of bias).
a Unless a published study protocol was available, completeness of reporting was judged
solely on correspondence of Results section with details in Methods section of article.
b Data not extractable for meta-analysis.
c Expected high risk of bias in domain IIIA—trial assessed as ‘(much) more pragmatic
than explanatory’.
d High risk of bias in domain IIIA only: trial is otherwise ‘Uncertain risk of bias’
overall.
There were no A- or B1*-rated trials. One trial (A283, Sinha) was rated uncertain risk of bias (‘B2’-rated). Ten trials were assessed as high risk of bias (‘C’-rated); one of those (A296, Peckham) was judged high risk of bias for domain IIIA only. Seven ‘C’-rated trials were deficient in three or more domains
of assessment (‘C3.0’-rated or worse). A summary risk-of-bias bar graph is shown in
[Supplementary File 5] (available in online version only).
Pragmatic/Explanatory Attitude
Independent assessment identified a total of 26 domains (one to six domains per trial)
for which the two scores were ≥ 2 apart. In all cases, consensus discussion narrowed
the score discrepancy per domain to no more than 1, which was our target. There was
no trial for which the consequent two total scores lay on either side of a threshold
for overall characterisation of pragmatic/explanatory attitude: a final designation
per trial was thus readily apparent ([Table 3]). Five of the 11 trials were more pragmatic than explanatory, two were more explanatory
than pragmatic and four were equally pragmatic and explanatory. All four of the trials
with study design Ib were more (or much more) pragmatic than explanatory in attitude.
Across all trials, each of 19 domains was given a score of 3 due to the absence of
sufficient information in the original article: this lack of information pertained
especially to domain 8 (participant compliance) and domain 9 (protocol adherence).
Meta-Analysis
The selected main outcome was not estimable for three studies (see [Supplementary File 4]), for which group SDs for the designated end-point could not be derived. For the
remaining eight studies, five were in study-design category Ia (Other-intervention
control) and three were in category Ib (‘[A + B] versus B’); there were no trials
in category II (No-treatment control). One category Ia study (A138, Adler) was a non-inferiority
trial and was examined separately.
Overall Relative Effect Sizes
For the superiority trials, it transpired that the distribution of dichotomous and
continuous main outcome measures was the same as that of trials with study design
Ia and Ib; the non-inferiority trial's main outcome measure was a continuous variable.
For a given study design, therefore, there was no need to transform/combine data from
effect estimates expressed as OR or SMD.
Study design Ia: Other-intervention control: The pooled effect estimate for all four superiority trials was OR = 0.87 (95% CI:
0.30 to 2.48; p = 0.79)—see [Fig. 2]. I
2 = 70%, indicating significant statistical heterogeneity between studies. There was
also very substantive clinical heterogeneity amongst the four studies, with conditions of investigation comprising
otitis media, vulvovaginal candidiasis and malaria. Three of the trials had a non-significant
effect favouring homeopathy; the fourth trial (A149, Witt) had a statistically significant
effect favouring the comparator.
Fig. 2 Forest plot for four analysable randomised controlled trials of individualised homeopathic
treatment (IHT): study design Ia (IHT versus other intervention). Abbreviations: CI, confidence interval; df, degrees of freedom;
M-H, Mantel-Haenszel method.
As shown in [Table 3] and summarised in [Fig. 2], three of the four trials displayed high risk of bias; the fourth trial (A283, Sinha)
had uncertain risk of bias. Three trials were equally pragmatic and explanatory in
attitude; the fourth trial (A149, Witt) was more explanatory than pragmatic.
With such similar attributes and small numbers of studies, we did not pursue sensitivity
analyses based on risk of bias or on external validity. Small numbers also made redundant
our planned sub-group analyses.
Non-inferiority trial (A138, Adler): The effect estimate for this single trial was
SMD = –0.41 (95% CI: –0.95 to 0.12; p = 0.13)—see [Fig. 3]. This finding, reflecting also the original authors' designated margin of non-inferiority,
is consistent with a conclusion that IHT was non-inferior to fluoxetine in the treatment
of depression for these subjects. The trial displayed high risk of bias due to its
39.6% patient attrition rate. Its study attitude was more explanatory than pragmatic.
Fig. 3 Forest plot for non-inferiority randomised controlled trial of individualised homeopathic
treatment (IHT): study design Ia (IHT versus other intervention). Abbreviations: CI, confidence interval; IV, inverse-variance
method; SD, standard deviation; Std., Standardised.
Study design Ib: ‘[A + B] versus B’: The pooled effect estimate for all three trials was SMD = –0.26 (95% CI: –0.47 to
–0.05; p = 0.02)—see [Fig. 4]. I
2 = 0%, indicating that heterogeneity was due to sampling error alone. Again, however,
there was very substantive clinical heterogeneity amongst the studies, with conditions of investigation comprising menopausal
syndrome, irritable bowel syndrome and cancer. Each of two trials had a non-significant
effect favouring adjunctive IHT; the third trial (A304, Frass) had a marginally statistically
significant effect favouring adjunctive IHT.
Fig. 4 Forest plot for three analysable RCTs of IHT: study design Ib (IHT + other intervention
versus other intervention alone). Abbreviations: CI, confidence interval; IV, inverse-variance
method; SD, standard deviation; Std., Standardised.
As shown in [Table 3] and summarised in [Fig. 4], each of the three trials displayed high risk of bias. The study attitude of each
was judged to be more (or much more) pragmatic than explanatory. One of the RCTs (A296,
Peckham) had high risk of bias in domain IIIa only: a sensitivity analysis would have
taken that factor into account.
As for the Ia trials above, we did not undertake sensitivity or sub-group analyses
because of the overall similarity of study quality and the small number of studies.
Disease- and Category-Specific Relative Effect Sizes
With 11 unique clinical conditions, it was not possible to carry out disease-specific
analyses. Ear Nose & Throat was the only category of condition for which data were
available from at least two studies: chronic otitis media (A141, Harrison) and acute
otitis media (A283, Sinha). The pooled effect estimate was OR = 1.28 (95% CI: 0.61
to 2.70; p = 0.52)—[Fig. 5]. It is thus inconclusive whether IHT and the comparator interventions differed in
effectiveness. These two trials had high/unclear risk of bias; in both, the study
attitude was equally pragmatic and explanatory.
Fig. 5 Forest plot for two RCTs of IHT: clinical category, Ear Nose and Throat. Abbreviations:
CI, confidence interval; df, degrees of freedom; M-H, Mantel-Haenszel method.
Discussion
None of the 11 studies was judged to comprise reliable evidence (i.e. there were no
A- or B1*-rated trials), 10 being assessed as high risk of bias. Three of the 11 articles
failed to yield data suitable for meta-analysis, leaving eight studies in three different
categories of study design: collectively for four heterogeneous trials with study
design Ia, the effects of IHT were inconclusive; for three clinically heterogeneous
studies with design Ib, there was a statistically significant effect favouring adjunctive
IHT; in the remaining trial IHT was found non-inferior to fluoxetine in the treatment
of depression. Lack of replication of subject matter prevented meta-analysis of IHT
by clinical condition, and our analysis by category of condition was limited to pooling
data from two trials of otitis media (chronic and acute disease), which found a non-significant
effect of IHT (pooled OR = 1.28; p = 0.52). The low intrinsic quality and number of studies preclude any clear conclusions
about comparative effectiveness of IHT. The same limitations prevented analysis of
any sub-group data. An additional literature search has revealed only two further
OTP-controlled trials that might have been eligible for inclusion in the current article.[34]
[35]
Within the pragmatic–explanatory continuum, the internal validity for pragmatic study
design would typically be lower than that for explanatory design.[36] Logically, therefore, an intrinsically low-quality OTP-controlled study might be
expected to have more emphasis placed on external validity; however, it is notable
that, despite their OTP-controlled design, only five of the studies possessed a truly
pragmatic study attitude. The external validity of six trials was therefore no better
than that of an overtly explanatory trial, with limited generalisability of findings
to a wider population of patients. The non-inferiority of IHT to fluoxetine in the
treatment of depression, observed in one trial (A138, Adler), was found to lack the
external validity that the original authors presumably intended. By contrast, the
studies that examined adjunctive homeopathy were all positioned within the pragmatic
band of the pragmatic–explanatory continuum, affirming generalisability of their findings.
Although the PRECIS tool was originally developed to help at the design stage of clinical trials,[22] we found it fairly straightforward to apply in assessing the pragmatic/explanatory
attitude of a completed trial. We recognise also that the empirical scoring notation we devised diverges
from the specific methods suggested in the original PRECIS article;[22] however, our approach has allowed us to approximate a given trial's positioning
on the pragmatic–explanatory continuum, and is in keeping with the original PRECIS authors' receptiveness to further methodological innovation.[22] The 10-domain judgmental criteria were operationalised successfully, though there
was often a lack of information in the published articles to form a clear opinion
on participant compliance (domain 8) and protocol adherence (domain 9). Our scoring
method may be viewed—like the original PRECIS authors' approach—as a ‘work in progress’.
The homeopathy community expresses reasonable concern about the sensitivity of the
placebo-controlled trial design to detect a treatment effect of its medicines.[37] Indeed, the wider Complementary/Alternative Medicine (CAM) community also recognises
that research on a ‘whole system of care’, such as IHT, should avoid focus solely
on ‘active ingredients’ (i.e. the efficacy of the medicine).[38] With its particular focus and emphasis, each type of study design in IHT is thus important: placebo-controlled design to examine
proof of concept that homeopathic medicines may have a specific treatment effect;
OTP-controlled design to evaluate that concept within a package of care that is everyday
homeopathy practice. Our meta-analysis of placebo-controlled RCTs of IHT suggested
evidence for a distinct effect of homeopathic medicines taken as individualised prescriptions;[2] our current analysis of OTP-controlled RCTs, however, has been less successful in
evaluating that concept due to the low quality and quantity of relevant studies.
Unless the ‘double-dummy’ approach is adopted, OTP-controlled trials have expected
high risk of bias regarding blinding of participants. In fact, our SR has highlighted
that low risk of bias in OTP research may be feasible only in double-dummy trials (A138, Adler; A148, van Erp); however, such trials also have
limited pragmatic attitude and a focus that is on the efficacy of the homeopathic
medicine rather than on the whole package of homeopathic care. By contrast, OTP-controlled
design may be optimal in studies of adjunctive IHT (‘[A + B] versus B’ design), where
we found a prevailing pragmatic attitude and, in one case (A296, Peckham), a high
risk of bias in domain IIIa only: our aim to accommodate that sole bias within sensitivity
analysis was thwarted by the low quality and number of relevant trials. Some observers
might argue, nevertheless, that the Cochrane-type system for rating internal validity
is inappropriate for more pragmatically orientated trials of ‘whole-system’ interventional
approaches such as homeopathy.
We emphasise that the extreme clinical heterogeneity of the current studies seriously
limits the interpretation of the corresponding meta-analyses. The pooled effect estimates
presented in [Fig. 2] and [Fig. 4] must be viewed with appropriate caution: indeed, these results are not represented
in Conclusions. It should also be noted that ‘[A + B] versus B’ trial design in individualised CAM
therapy has been criticised as being prone to ‘false positive’ results due to ‘non-specific
effects’ of the intervention.[39] Any issues concerning model validity of these current trials—a key attribute, as
previously emphasised[38]
[40]—are unknown at present.
Conclusions
Due to the low quality, the small number and the extreme heterogeneity of studies,
no decisive conclusion is currently possible regarding comparative effectiveness of
IHT on health-related outcomes, assessed within the context of OTP-controlled trials.
Generalisability of findings is limited by the variable extent of external validity
observed overall. To date, the highest quality and the most pragmatic study attitude
for RCTs of IHT is associated with the ‘[A + B] versus A’ approach. Future OTP-controlled
trials in homeopathy should promote both internal validity and external validity,
as informed by the domain-based assessments in the present article.
