British Homeopathic Association : News

The article published on homeopathy in the Lancet recently has fundamental flaws, writes Robert Mathie, Research Development Adviser of the BHA and Faculty of Homeopathy.

What the paper studied:

The authors analysed placebo-controlled trials of homeopathy and conventional medicine (matched for disorder and type of clinical outcome) published up to January 2003 (1). 110 homeopathy trials and 110 matched trials in conventional medicine were included; they were not necessarily peer-reviewed papers. They were assessed using standard criteria of trial quality (randomisation, masking, data analysis methods) and by treatment effect using odds ratios (an odds ratio of less than 1.0 indicating an effect greater than placebo). The original studies included a wide range of medical conditions; nearly half the studies comprised research in respiratory tract infections, pollinosis & asthma, and gynaecology & obstetrics. The homeopathy trials concerned all forms of the therapy, including classical (individualised) homeopathy, ‘clinical homeopathy’, complex homeopathy, or isopathy. The majority of conventional medicine trials investigated specific pharmaceutical drugs (non-steroidal, anti-allergic, virostatic, or antibiotic). Treatment approach in each category of trial was either therapeutic or prophylactic.

What the paper shows:

Viewing the results of the trials overall, there is a broadly similar positive treatment effect in both homeopathy and conventional medicine. In both categories of trial, those with fewer patients typically showed more positive treatment effects than larger trials. This is shown in the paper’s Figure 2, where trials with the largest standard error tend to have the lowest odds ratio. The authors do not quote mean odds ratios for the two groups of trials. 21 homeopathy trials and 9 in conventional medicine were judged to be of ‘higher quality’. The author’s key analysis was then restricted to the ‘larger’ 14 of those trials (8 homeopathy and 6 conventional medicine; none of them cited specifically); this analysis resulted in a mean odds ratio of 0.88 for homeopathy trials and 0.58 for conventional medicine trials. There was thus no longer a convincing positive treatment effect of homeopathy as compared with that of conventional medicine. The paper’s main conclusion is that the clinical effects of homeopathy are probably those of placebo.

What the Lancet editor says:

Under the headline ‘The end of homoeopathy’, the journal editorial states ‘Now doctors need to be bold and honest with their patients about homoeopathy’s lack of benefit, and with themselves about the failings of modern medicine to address patients’ needs for personalised care’. It adds the comment ‘Surely the time has passed for selective analyses, biased reports, or further investment in research to perpetuate the homoeopathy versus allopathy debate’.

Our commentary on this publication:

As the following paragraphs illustrate, the paper has not demonstrated homeopathy’s lack of benefit. Regrettably, in publishing and commenting on this paper, the journal has displayed some of these unwelcome attributes of selective analysis and biased reporting. And investment in clinical research in homeopathy needs to be enhanced, not withheld.

There are a number of concerns in the way the paper approaches homeopathy trials: for example, its criteria of study quality do not reflect the homeopathic relevance of the clinical outcome/s measured; in addition, placebo-controlled design was probably not appropriate in the trials of individualised homeopathy (2). In other words, standard assessment criteria are insufficient to gauge ‘high quality’ in homeopathy trials.

Another fundamental concern is that the paper gives no clue about the nature of the 14 trials selected for the key analysis: whether they were mainly therapeutic or prophylactic, for example, and whether the homeopathic interventions were classical, ‘clinical’ or complex homeopathy, or isopathy. Knowledge of these would potentially make a great difference to the inferences that should be drawn. Given the heterogeneity of homeopathy trials, it seems unlikely that the design and methods of just 8 can be representative of 110. Nor are we offered proper summary data on the odds ratios for effectiveness in the two sets of 110 trials overall; without such information, it is impossible to gauge the impact of having narrowed the analysis to just 8+6 trials.

There are other bizarre features of this paper. On reading its text, it seems that only the literature between 1995 and January 2003 was included in the analysis. This would have built on a previous major meta-analysis of homeopathy trials (3). However, on examination of the web-table that lists all the references, it becomes apparent that 62 of the papers analysed were actually published before 1995. The remaining papers analysed were published from that year onwards, but some of the main articles during that time have not been included. Inexplicably too, a substantial number of the papers reviewed in the previous meta-analysis are absent from the new one.

The wider view:

Most independent scientific observers would regard this analysis as inconclusive in its results and opaque in some of its key methods and reporting. And it has the limitations of any analysis of clinical research in homeopathy that attempts to group together all homeopathic conventions of treatment and all medical conditions that have been investigated. A comprehensive analysis of that type can merely make overall conclusions and may miss specific areas of therapeutic importance – the authors themselves highlight (but dismiss) the fact that 8 trials of homeopathy in upper respiratory tract infections have strongly positive findings overall. It is for this very reason that we adopted an analysis of the homeopathic research literature that focused instead on individual clinical trials and their findings (reference 10 in ‘Research in Homeopathy’ section of this website) (4).

The way forward:

We certainly agree with the Lancet paper’s authors that ‘future research efforts should focus on … the place of homeopathy in health-care systems’ and about ‘the failings of modern medicine’. That should mean concentrating more on conducting trials that compare homeopathy properly and fairly with standard medical care. There remains a place for placebo-controlled trials, but these have to be considered with insight and wisdom. These and other trial design issues in homeopathy should be properly informed by prior clinical observational research and well-conceived pilot trials. The Lancet publication has done nothing to inform this important field of research in the constructive or careful manner that it deserves.

References:

1. Shang A, Huwiler-Müntener K, Nartey L, Jüni P, Dörig S, Sterne JAC, Pewsner D, Egger M. Are the clinical effects of homeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy. Lancet 2005; 366: 726-32.
2. Weatherley-Jones E., Thompson EA, Thomas KJ. The placebo-controlled trial as a test of complementary and alternative medicine: observations from research experience of individualised homeopathic treatment. Homeopathy 2004; 93: 186-9.
3. Linde K, Clausius N, Ramirez G, Melchart D, Eitel F, Hedges LV, Jonas WB. Are the clinical effects of homeopathy placebo effects? A meta-analysis of placebo-controlled trials. Lancet 1997; 350: 834-43.
4. Mathie RT. The research evidence base for homeopathy: a fresh assessment of the literature. Homeopathy 2003; 92: 84-91.

Printer friendly view

	Members of the public What is homeopathy? Find a practitioner FAQs Homeopathy in the NHS Articles about homeopathy Healthcare professionals Overview Education and training Membership of the Faculty Research in Homeopathy Table of contents Meta-analyses Clinical research Outcome studies Our research strategy		Lancet article is fundamentally flawed 07-09-2005, 3:02 pm HOME News and Press Releases Printer friendly view The article published on homeopathy in the Lancet recently has fundamental flaws, writes Robert Mathie, Research Development Adviser of the BHA and Faculty of Homeopathy. What the paper studied: The authors analysed placebo-controlled trials of homeopathy and conventional medicine (matched for disorder and type of clinical outcome) published up to January 2003 (1). 110 homeopathy trials and 110 matched trials in conventional medicine were included; they were not necessarily peer-reviewed papers. They were assessed using standard criteria of trial quality (randomisation, masking, data analysis methods) and by treatment effect using odds ratios (an odds ratio of less than 1.0 indicating an effect greater than placebo). The original studies included a wide range of medical conditions; nearly half the studies comprised research in respiratory tract infections, pollinosis & asthma, and gynaecology & obstetrics. The homeopathy trials concerned all forms of the therapy, including classical (individualised) homeopathy, ‘clinical homeopathy’, complex homeopathy, or isopathy. The majority of conventional medicine trials investigated specific pharmaceutical drugs (non-steroidal, anti-allergic, virostatic, or antibiotic). Treatment approach in each category of trial was either therapeutic or prophylactic. What the paper shows: Viewing the results of the trials overall, there is a broadly similar positive treatment effect in both homeopathy and conventional medicine. In both categories of trial, those with fewer patients typically showed more positive treatment effects than larger trials. This is shown in the paper’s Figure 2, where trials with the largest standard error tend to have the lowest odds ratio. The authors do not quote mean odds ratios for the two groups of trials. 21 homeopathy trials and 9 in conventional medicine were judged to be of ‘higher quality’. The author’s key analysis was then restricted to the ‘larger’ 14 of those trials (8 homeopathy and 6 conventional medicine; none of them cited specifically); this analysis resulted in a mean odds ratio of 0.88 for homeopathy trials and 0.58 for conventional medicine trials. There was thus no longer a convincing positive treatment effect of homeopathy as compared with that of conventional medicine. The paper’s main conclusion is that the clinical effects of homeopathy are probably those of placebo. What the Lancet editor says: Under the headline ‘The end of homoeopathy’, the journal editorial states ‘Now doctors need to be bold and honest with their patients about homoeopathy’s lack of benefit, and with themselves about the failings of modern medicine to address patients’ needs for personalised care’. It adds the comment ‘Surely the time has passed for selective analyses, biased reports, or further investment in research to perpetuate the homoeopathy versus allopathy debate’. Our commentary on this publication: As the following paragraphs illustrate, the paper has not demonstrated homeopathy’s lack of benefit. Regrettably, in publishing and commenting on this paper, the journal has displayed some of these unwelcome attributes of selective analysis and biased reporting. And investment in clinical research in homeopathy needs to be enhanced, not withheld. There are a number of concerns in the way the paper approaches homeopathy trials: for example, its criteria of study quality do not reflect the homeopathic relevance of the clinical outcome/s measured; in addition, placebo-controlled design was probably not appropriate in the trials of individualised homeopathy (2). In other words, standard assessment criteria are insufficient to gauge ‘high quality’ in homeopathy trials. Another fundamental concern is that the paper gives no clue about the nature of the 14 trials selected for the key analysis: whether they were mainly therapeutic or prophylactic, for example, and whether the homeopathic interventions were classical, ‘clinical’ or complex homeopathy, or isopathy. Knowledge of these would potentially make a great difference to the inferences that should be drawn. Given the heterogeneity of homeopathy trials, it seems unlikely that the design and methods of just 8 can be representative of 110. Nor are we offered proper summary data on the odds ratios for effectiveness in the two sets of 110 trials overall; without such information, it is impossible to gauge the impact of having narrowed the analysis to just 8+6 trials. There are other bizarre features of this paper. On reading its text, it seems that only the literature between 1995 and January 2003 was included in the analysis. This would have built on a previous major meta-analysis of homeopathy trials (3). However, on examination of the web-table that lists all the references, it becomes apparent that 62 of the papers analysed were actually published before 1995. The remaining papers analysed were published from that year onwards, but some of the main articles during that time have not been included. Inexplicably too, a substantial number of the papers reviewed in the previous meta-analysis are absent from the new one. The wider view: Most independent scientific observers would regard this analysis as inconclusive in its results and opaque in some of its key methods and reporting. And it has the limitations of any analysis of clinical research in homeopathy that attempts to group together all homeopathic conventions of treatment and all medical conditions that have been investigated. A comprehensive analysis of that type can merely make overall conclusions and may miss specific areas of therapeutic importance – the authors themselves highlight (but dismiss) the fact that 8 trials of homeopathy in upper respiratory tract infections have strongly positive findings overall. It is for this very reason that we adopted an analysis of the homeopathic research literature that focused instead on individual clinical trials and their findings (reference 10 in ‘Research in Homeopathy’ section of this website) (4). The way forward: We certainly agree with the Lancet paper’s authors that ‘future research efforts should focus on … the place of homeopathy in health-care systems’ and about ‘the failings of modern medicine’. That should mean concentrating more on conducting trials that compare homeopathy properly and fairly with standard medical care. There remains a place for placebo-controlled trials, but these have to be considered with insight and wisdom. These and other trial design issues in homeopathy should be properly informed by prior clinical observational research and well-conceived pilot trials. The Lancet publication has done nothing to inform this important field of research in the constructive or careful manner that it deserves. References: 1. Shang A, Huwiler-Müntener K, Nartey L, Jüni P, Dörig S, Sterne JAC, Pewsner D, Egger M. Are the clinical effects of homeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy. Lancet 2005; 366: 726-32. 2. Weatherley-Jones E., Thompson EA, Thomas KJ. The placebo-controlled trial as a test of complementary and alternative medicine: observations from research experience of individualised homeopathic treatment. Homeopathy 2004; 93: 186-9. 3. Linde K, Clausius N, Ramirez G, Melchart D, Eitel F, Hedges LV, Jonas WB. Are the clinical effects of homeopathy placebo effects? A meta-analysis of placebo-controlled trials. Lancet 1997; 350: 834-43. 4. Mathie RT. The research evidence base for homeopathy: a fresh assessment of the literature. Homeopathy 2003; 92: 84-91. Printer friendly view
			Home \| Faculty of Homeopathy \| British Homeopathic Association \| Research in homeopathy © 2000 - 2007 British Homeopathic Association. Site developed by Chord9 Ltd.