The methodological quality of trials evaluating the effectiveness of acupuncture in the treatment of chemotherapy-induced peripheral neuropathy
Peripheral Neuropathy (PN) is a side effect for patients undergoing certain types of chemotherapy treatment. This results in dysfunction of sensory, motor and or autonomic neurons of the peripheral nervous system, Postma & Heimans (2000) noted. The presentation of CIPN varies widely. The pain sensation can range from numbness and/or tingling, to shooting pain and can change the perception and sensation of the affected limbs. Some patient’s symptoms are aggravated by rest, some by activity. Differences in temperature can often be a triggering factor too.
PN poses a major problem for patient care as pharmacological treatments are often inadequate, Dworkin, et al. (2007) described. This is due to dosing complexities, delayed analgesic onset and common side effects of CIPN treatments such as comnolence. In some types of chemotherapies such as Bortezomib, Thalidomide and Oxaliplatin the PN can become dose limiting factor.
Preliminary acupuncture studies in the management of diabetes and HIV induced PN could demonstrate effect in improving the symptoms using acupuncture alone as well as improving the outcome of treatment when combined with standard medication. This is taken by researchers as a reason to explore acupuncture in the management of chemotherapy induced PN.
The mechanism of acupuncture in the treatment of PN is unclear. Acupuncture is associated with a number of biochemical mediators like Y-aminobutyric acid, glutamate, adenosine and Beta-endorphin, which are all active within the nervous system, Zhao (2008) and Goldman, et al. (2010) found.
This paper is going to explore the methodology and content of a non-comprehensive number of papers written about the efficacy of acupuncture in the treatment of chemotherapy induced peripheral neuropathy (CIPN) and will in the second half of the paper discuss the methodological quality of a randomized control trial (RCT) that was conducted in the indication.
To find the papers on CIPN, a literature search between January and April 2014 was conducted in the databases Cinahl, Medline, Science direct and Google Scholar. The search terms used were *Neuropathy*, *Neuropathic*, *Acupuncture*, *Electro?Acupuncture* and the German equivalents *Akupunktur* *Elektro?akupunktur*, *Neuropathie* and *Neuropathisch*. A search in the reference list of the Franconi review (2013) was undertaken too. I considered only full texts evaluating the efficacy or effectiveness of acupuncture in CIPN in humans. I discarded all papers that required payment. Papers in all languages other than German or English were discarded.
Overall, using the criteria described above, I found six papers to discuss. The list is not comprehensive. The papers are summarized in Appendix 1.
Value of the evidence demonstrated in the papers
While evaluating the quality of the papers found in the context of the pyramid of evidence, it became obvious that the majority of papers found are at the bottom end for consideration in evidence based practice.
I have included in this review a case series by Wong & Sagar (2006) of purely anecdotal informative value. This is also the case with a case report by Bao, et al. (2011). The two pilot studies, by Irnich, et al. (2002) and Schroeder, et al. (2012) suffer from very small sample sizes and are not sufficiently powered. They were conducted to test potential trial designs in the indication. One of the papers is a retrospective service evaluation. The most valuable of the papers – at least from the perspective of the pyramid of evidence – is the randomized controlled trial conducted by Schroeder, et al. (2012). I will evaluate its methodology in the second half of this essay.
The quality of the evidence of the reported results has to be evaluated as low due to the methodology and low sample size of the papers. It is not unusual to find for the use of acupuncture in most conditions only articles about acupuncture efficacy of anecdotal quality. This is due to the relative youth of acupuncture research in the western world and the difficulty of testing acupuncture’s effect in a trial setting.
Evidence-based practice requires significant numbers of patients and practitioners, their blinding and the use of suitable placebos and comparators. All of this leads to the need of significant funding.
Evaluation and comparison of acupuncture treatments
Due to the small size and low quality of reported evidence I will concentrate on the concrete treatment interventions tested. This involves examining which points are used and why, with which theoretical background they are chosen and whether treatment is standardized or individualized. We also need to examine the training background of the practitioners providing the treatment and the context in which the treatment was done.
The list above is supported by the STRICTA recommendations (Standards for reporting interventions in controlled trials of acupuncture), which were devised by MacPherson, et al. (2002). These recommendations were devised by a group of acupuncturists representing universities and schools from different countries, who met to develop guidelines that journals could use to evaluate the quality of an acupuncture paper.
The following table outlines their recommendations:
MacPherson, et al. (2002) underline the importance of outlining the style of acupuncture given and the rationale for treatment, preferably with referenced sources. The details of the chosen points and needles must be provided too.
None of the papers discussed in this review are reporting on the acupuncture rationale and the practitioner background. The other aspects are referred to insufficiently.
The majority of acupuncture trials suffer from poor methodology in determining (or at least explaining) point prescriptions (the choice of a set of points for needling). None of the papers cited in this review explain their methodology behind the choice of point prescription.
Acupuncture trials might benefit a great deal by adopting approaches used by Western medical research. RCTs, for example, which are very commonly used in Western medicine, use an at least 3-tiered structure. Simplifying the description of this process it can be said that when an effective compound is found –and tested sufficiently in animals, the substance is tested for toxicity among a very small group of human subjects. In the second instance, its effectiveness in different dosage levels is tested. Finally, a large RCT is conducted in which the compound’s efficacy is compared with the current standard of care.
Similar rigour could be applied to acupuncture research. Points could be chosen by asking a large sample of acupuncture practitioners to outline their current treatment approach in the defined indication (and choosing that which is most frequently used), and by gathering a panel of experts in the indication who will be tasked with recommending a particular points prescription. Using such a systematic framework to generate treatment protocols would lend greater transparency to acupuncture research. This approach was already used in researching acupuncture treatments for back pain, but is rarely used in researching treatments in other indications.
The practitioners‘ backgrounds
STRICTA recommends that the practitioners‘ backgrounds should be described. This is due to the widely varying level of education that acupuncturists receive in different countries due to different regulatory frameworks.
In the UK a degree (3-4 years) from an institute verified by the British Acupuncture Council, the relevant UK regulatory body, is required. In Germany, Switzerland and Austria, a medical degree is required, followed by an acupuncture course lasting 200 hours (120 hours of theory, 80 hours of practice). This allows doctors then to practice acupuncture which is reimbursed by the health insurance companies. In the USA, students study a 3-4 year long degree level course which has to be approved by the National Certification Commission for Acupuncture and Oriental Medicine. In China it is common practice to be trained in Chinese Medicine as well as Western Medicine which results in a training time of 6-7 years.
The practitioners themselves are a significant variable in affecting treatment outcomes in acupuncture. Therefore their background should be properly accounted for.
In the papers at hand the practitioners‘ backgrounds are not accounted for. It would be interesting to know which school of TCM they studied and how long they have practised for.
There is a wide variety of point prescriptions used in the studies presented in this review. On the right side you can see the points used in the standardized point prescription which Wong & Sagar, (2006) used in their case study.
The points are Bafeng (between the toes), Baxie (between the fingers), LI11 (a point at the lateral side of the elbow, that has the function to clear heat, from the body and from the blood – chemotherapy is said to induce toxic heat into the body) and ST36 (the red dot below the knee on the lateral side of the tibia). The ST36 point is indicated in tonifying Blood and Qi and is the equivalent to LI11 on the lower leg. CV6, the dot below the belly button, governs Qi.
The treatment is standardized. Bafeng and Baxie are point combinations that are very commonly used in the treatment of neuropathy in china. Unfortunately Wong and Sagar fail to explain their reasoning behind the use of these points.
The study of acupuncture treatments into CIPN is, for obvious reasons, not referenced in classical Chinese acupuncture texts, a key source for acupuncture practitioners.
Bob Flaws, a UK-based expert in traditional Chinese medicine recommended Bafeng and Baxie in the treatment of neuropathy (Flaws & Sionneau, 2005).
The image on the right shows the distribution of points used in a patient with CIPN in the lower extremities, who was treated during the case study from Bao, et al., (2011). The points between the toes called Bafeng are combined with ST40, a point indicated in case of the presentation of phlegm, LI11, used in the presentation of Heat, SJ5, a calming point which clears Wind and Heat, and LI4, an important point in the management of pain.
Interestingly, there is a lot of overlap in the two point prescriptions shown so far. Unfortunately, the methodology behind the selection of these points is not outlined in Bao, et al.’s study either.
The trial of Rostock, et al., (2013) uses Electro-Acupuncture. This is the randomized control trial which is going to be discussed regarding its methodological quality in the second part of this review.
Unfortunately, this paper does not outline on which of points the EA is used (which points were connected.) The study was unsuccessful in demonstrating efficacy. This study suffers from an especially short duration (shortest of all the studies), of only 3 weeks.
Schroeder, et al.’s (2012) trial was one of the smallest in our sample, with only six patients treated. In this study, the patients‘ peripheral neuropathy in the lower extremities was treated using the points presented in the image on the left. This is a particularly painful point prescription. The points used are, again, Bafeng (between the toes), as well as another group of points at the tips of the toes. I am not sure if this standardized treatment could be handled by all types of CIPN sufferers. Some CIPN sufferers suffer from increased sensitivity (and severe pain) as a result of the condition, potentially making the treatment recommended by Schroeder, et al. unsuitable. The treatment protocol might suit those patients whose CIPN is characterized by numbness (and severe pain).
This point prescription differs considerably from the ones outlined above and varies considerably from the protocols associated with traditional Chinese medicine.
Irnich, et al., (2002) conducted a study using a differentiated point protocol. They used points that were chosen along TCM diagnosis and pattern differentiation. The red points in the diagram are the points most commonly used. The green points show other points that were used while the treatments. Interestingly, Bafeng and Baxie, the point combinations between toes and fingers commonly used in the other studies, are not used in this protocol. The authors fail to explain why they did not consider these points in their treatment protocol. It is possible that the acupuncturists involved with the study were concerned with the pain this treatment might inflict on their patients. Schroeder, et al., (2012), seemed more comfortable to let the patients deal with some discomfort.
It is notable that in most of the studies outlined above, practitioners failed to give precise reasons behind their choice of treatment protocols. It is impossible to know for certain the reasons behind this exclusion, but it is possible that these practitioners‘ own unstated preferences and prejudices played a part. These may have come about as a consequence of these researchers‘ considerable experience, or not, but the fact that these are not stated harms the quality of the research regardless.
It is true that there is considerable overlap in some of the points used (Bafeng, Baxie, ST36 and LI11 in particular). While this hints at an unspoken consensus among the acupuncturist community in the treatment of CIPN, further research into the treatment of the condition would greatly benefit from a more systematic approach in the generation of point prescriptions.
In the second part of this paper I am going to discuss the methodological quality of the randomized control trial from Rostock et al., (2013).
PART 2: Methodological problems with one research piece.
Chemotherapy-induced peripheral neuropathy in cancer patients: a four arm randomized trial on the effectiveness of electro acupuncture
The highest value of evidence according to the evidence pyramid in my systematic review can be found in Rostock, et al.’s (2013) RCT.
This trial reported negative results. Unusual for a medical research paper, it was published despite revealing no positive effect and despite having a relatively small sample size. With the quality of research being harmed by the fact that many papers revealing negative or inconclusive results are never published, the fact that this one was published is notable. The fact that trials with small sample sizes are particularly unlikely to be published if the results are inconclusive or negative, leads to the literature being flooded by positive results. This is a big problem for meta-analysis, which is dependent on the results of many small trials which was noted by Dwan, et al., (2008).
The group sequential adaptive design is considered to be a particularly ethical research design. „Adaptive,“ in this context, means that there is an interim analysis which can lead, if the results are sufficiently positive, to an early trial stop and the possibility for all participants to receive an effective treatment sooner. „Sequential“ means, in this context, that if the trial would have been continued, then groups could have been swapped. The participants in the different treatment groups could have become their own controls.
Rostock et al., (2013) used very simple patient reported measures for pain, instead of a validated score. They used a 10 point visual scale to measure the CIPN symptoms of the participants.
Patients were asked to describe how heavily they suffered at various points in the study and to rate the severity of neuropathic symptoms on a numerical rating scale (NRS)-ranging from 0 (no complaints) to 10 (highest imaginable complaints). The change from day 0 to day 21 on this patient-reported numerical rating scale was the primary outcome of the study.
It must be noted that this is not a validated pain score as for example the multiple sclerosis severity score (MSSS), which is considered a more objective measure. The manifestation of CIPN varies widely between patients which leads among other things to problems with this outcome measure. This will be discussed later.
The trial is sufficiently powered, which is positive. The authors report a type II error of 16.7% associated with trial design. Therefore the power is 83.3%, which is a high level commonly used by pharmaceutical companies. 83.3% offers a sufficiently high likelihood of detecting effect and offers the researchers a large likelihood of coming to a conclusive result. Trials which are low powered lead to inconclusive results if they find nothing. It raises the likelihood of patients taking part in a useless and potentially dangerous trial, as well as being a waste of resources.
Their statistical test is very simple. The participants‘ baseline pain levels are reported at the beginning. After three weeks the participants report again. Based on that outcome, the statistical test measures the change in the different treatment groups. There is no attempt made to control for potential confounders such as „acupuncture only works in women“. Indeed, there is a big gender imbalance in the different treatment groups. Typical other confounders are age, chemotherapy and cancer type. Such potential confounders should be taken into account.
Confounders were not a large concern in this trial, because the randomized allocation of treatment should theoretically break any associations between the treatment groups and any potential known or unknown confounders.
Unfortunately, due to the small size of the trial and despite the randomization, spurious associations may arise. Ideally the authors should have controlled for confounders using a multilevel regression model. They could have performed a linear regression of the change from baseline as the dependent variable and included potential confounders such as age, gender and cancer type in the model alongside treatment effect.
There might be one big confounder that Rostock, et al., might not have thought about: They should have adjusted for baseline pain levels. Baseline levels are very often associated with change from baseline: A person who had a lot of pain at the beginning is likely to indicate a much bigger improvement in comparison to someone who had relatively low pain levels at the beginning, even though both had the same reduction in pain levels overall.
Patients seem to report greater improvements the worse the level of suffering pre-treatment. This is a factor I regularly witnessed at the acupuncture clinic where we used mymop forms with 5 point likert scales to evaluate treatment outcome.
In this trial, this might have led to patients with lower baseline pain scores, diluting the greater effects that treatment might have had in the patients who had higher baseline pain scores.
The expression of CIPN might be another confounder. Gebera, et al., (2013) note that CIPN varies strongly in character and the pain levels cannot be predicted by the type of chemotherapy which further complicates the study. Some patients tend to report higher pain when lying or sitting down, for example. Other patients report higher pain when walking or standing. The failure to control for this important variable is a key problem with this study.
I do not believe that the expression of CIPN led to a variation in response to treatment but I am concerned that it affected the reporting of outcome on the ten point scale.
The trial further suffers from a small sample size: only 60 patients participated in the study, therefore randomization might have not been sufficient in controlling for confounders.
Overall I question if one can put three types of CIPN together in one sample, distribute them to four groups with about fifteen members each, measure their outcome along a ten point scale and make this the primary outcome measure of a study while stating that the statistical power of the trial is theoretically sufficient to prove the effect of electro-acupuncture.
PP vs ITT
The trial was done according to intention to treat (ITT) and not per protocol (PP). PP means that only those patients that took part in the whole trial are considered in the final analysis. In ITT every patient is considered, including those that dropped out. In ITT if you exclude patients that did not adhere to treatment then you could bias the results. They started off with a sample population of 60 people and finished with 50, representing a drop-out rate above 10%. In this ITT trial everybody is included in the final analysis. This can lead to patients diluting the results of that treatment arm where, for example, two people did not take part in the analysis at day 21 (VitB).
According to this trial design, the doctors used the last set of data for each patient in the interim analysis. Measurements were taken at baseline (day 0), at the interim analysis (day 21) and at follow up (day 84). Patients that did not adhere to treatment and drop-outs had their baseline measurement taken into the analysis.
In other studies, the problem with drop-outs is dealt with by estimating their likely treatment outcome (based on the treatment outcomes of similarly-affected patients), and including these theoretical results in the final study outcome. In this study, however, the authors included the drop-outs in the final study outcome, but did not ascribe to them any improvement following treatment. This decision ensured that the study reported more negative outcomes than it might have otherwise.
The authors say they used the last observation carried forward which means they used in at least four of those eight patients baseline value for the imputation of the missing data.
The authors are careful to note similarities in their demographic characteristics and cancer types of the participants. They support this argument with statistical tests of distributional differences of these variables, but these are rendered questionable by the small sample size (60 patients). As noted earlier, there is a considerable gender imbalance in the study. The effectiveness of the study might have been further diluted by the fact that the authors failed to control for different disease types in their study, which included patients with a wide variety of cancers and chemotherapies. A study which took into account such differences might have been able to provide useful information about the efficacy of acupuncture treatment in dealing with the side effects of specific chemotherapies, rather than chemotherapy in general. In the future a wide indication for acupuncture in the treatment of CIPN would be preferred of course but the heterogeneity of the study population raises the possibility that possible confounders are not being found.
The study also illustrates the importance of devising appropriate study protocols from the beginning of the study. The authors‘ commitment to an adaptive approach (ie. – their commitment to abandon the study after three weeks if treatment looked ineffective) meant they were unable to gather the more complete data that might have been obtained if the study had run for the maximum six weeks. In addition, their commitment to include drop-outs in the analysis at day 21 might have diluted the positive results gathered from the other patients. Science, as Einstein said, is shit.
CIPN is a condition that is very hard to treat. We are talking about patients that survived cancer but received large amounts of neurotoxic chemotherapies in order to do so. These patients are sick, weak, fundamentally poisoned and suffer now from a condition that is highly complicated to treat for any medicine. CIPN lasts for months to years and in some patients forever. The degree of CIPN the patient will suffer from is unpredictable while administering the chemotherapy.
The treatment durations of the case studies discussed in the systematic review were all at least six weeks long. Often a first course of treatment (4-6 weeks) was given, followed by a break and a second set of treatment. (See Appendix 1).
In the Rostock et al. trial the treatment modalities were tested for three weeks before the study was abandoned. Measuring the effect of the effect of an electro-acupuncture protocol in CIPN after three weeks is a waste of resources considering the severity of the indication.
A further three weeks of study, as prescribed if the treatment proved effective, would have at least yielded significant results for the study of acupuncture treatment of CIPN.
The research base in acupuncture treating CIPN in English and German language is very thin. Trials in the indication would benefit from larger samples, a systematic approach to generating point prescriptions, the reporting according to the STRICTA guidelines and more fundamental research into the mechanism of action of acupuncture. I would like to see more trials that do not compare western medical treatment modalities with acupuncture but who combine them with each other and compare with placebo, acupuncture alone and standard of care alone. Acupuncture does not want to be an alternative medicine, but a complementary treatment modality. It would be nice if future trials research acupunctures role not just as an alternative but complementary to the standard of care.
Bao, T., Zhang, R., Badros, A. & Lao, L., 2011. Acupuncture Treatment for Bortezomib-Induced Peripheral Neuropathy: A Case Report. Pain Research and Treatment, Volume 2011.
Donald, G., Tobin, I. & Stringer, J., 2011. Evaluation of acupuncture in the management of chemotherapy-induced peripheral neuropathy. Acupuncture in Medicine, 29(3), pp. 230-233.
Dworkin, R., O’Connor, A. & Backonja, M., 2007. Pharmacologic management of neuropathic pain: evidence-based recommendations. Pain, Issue 132, pp. 237-51.
Flaws, B. & Sionneau, P., 2005. The treatment of modern western medical diseases with chinese medicine. 2nd ed. Boulder: Blue poppy Press.
Franconi, G., Manni, L., Schröder, S. & Marchetti, R. N., 2013. A Systematic Review of Experimental and Clinical Acupuncture in Chemotherapy-Induced Peripheral Neuropathy. Evidence-Based Complementary and Alternative Medicine.
Goldman, N., Chen, M. & Fujita, T., 2010. Adenosine A1 receptors mediate local nociceptive effects of acupuncture. Nature Neuroscience, Issue 157, pp. 9-17.
Ho, P., Peterson, P. & Masoudi, F., 2008. Evaluating the Evidence: Is There a Rigid Hierarchy?. Circulation, Issue 118, pp. 1675-1684.
Irnich, D., Winklmeier, S., Beyer, A. & Peter, K., 2002. Elektrostimulationsakupunktur bei peripheren neuropathischen Schmerzsyndromen. Der Schmerz, Issue 16, pp. 114-20.
MacPherson, H. et al., 2002. Standards for reporting interventions in controlled trials of acupuncture: the STRICTA recommendations. Acupuncture in Medicine, 1(20), pp. 22-25.
Postma, T. & Heimans, J., 2000. Grading of chemotherapy induced peripheral neuropathy.. Annals of Oncology, Issue 11, pp. 509-13.
Rostock, M. et al., 2013. Chemotherapy-Induced Peripheral Neuropathy in Cancer Patients: A Four-Arm Randomized Trial on the Effectiveness of Electroacupuncture. Evidence-Based Complementary and Alternative Medicine.
Schroeder, S., Meyer-Hamme, G. & Epplee, S., 2012. Acupuncture for chemotherapy-induced peripheral neuropathy (CIPN): a pilot study using neurography. Acupuncture in medicine – BMJ Journals, Issue 30, pp. 4-7.
Wong, R. & Sager, S., 2006. Acupuncture treatment for chemotherapy-induced peripheral neuropathy – a case series. Acupuncture in Medicine , Issue 24, pp. 87-91.
Zhao, Z., 2008. Neural mechanism underlying acupuncture analgesia. Progress in Neurobiology, Volume 85, pp. 355-75.
Image used for the presentation of the point prescriptions: Anon., n.d. http://www.medicalj.ru [Online] Available at: medicalj.ru/images/stories/signssymptoms_male.jpg
[Accessed 5 April 2014].