Is surgical intervention more effective than non-surgical treatment for carpal tunnel syndrome? a systematic review
Journal of Orthopaedic Surgery and Research volume 6, Article number: 17 (2011)
Carpal tunnel syndrome is a common disorder in hand surgery practice. Both surgical and conservative interventions are utilized for the carpal tunnel syndrome. Although certain indications would specifically indicate the need for surgery, there is a spectrum of patients for whom either treatment option might be selected. The purpose of this systematic review was to compare the efficacy of surgical treatment of carpal tunnel syndrome with conservative treatment
We included all controlled trials written in English, attempting to compare any surgical interventions with any conservative therapies. We searched Cochrane Central Register of Controlled Trials (The Cochrane Library Issue 1, 2010), MEDLINE (1980 to June 2010), EMBASE (1980 to June 2010), PEDro (searched in June 2010), international guidelines, computer searches based on key words and reference lists of articles. Two reviewers performed study selection, assessment of methodological quality and data extraction independently of each other. Weighted mean differences and 95% confidence intervals for patient self-reported functional and symptom questionnaires were calculated. Relative risk (RR) and 95% confidence intervals for electrophysiological studies and complication were also calculated.
We assessed seven studies in this review including 5 RCTs and 2 controlled trials. The methodological quality of the trials ranged from moderate to high. The weighted mean difference demonstrated a larger treatment benefit for surgical intervention compared to non surgical intervention at six months for functional status 0.35( 95% CI 0.22, 0.47) and symptom severity 0.43 (95% CI 0.29, 0.57). There were no statistically significant difference between the intervention options at 3 months but there was a benefit in favor of surgery in terms of function and symptom relief at 12 months ( 0.35, 95% CI 0.15, 0.55 and 0.37, 95% CI 0.19 to 0.56). The RR for secondary outcomes of normal nerve conduction studies was 2.3 (95% CI 1.2, 4.4), while RR was 2.03 (95% CI 1.28 to 3.22) for complication, both favoring surgery.
Both surgical and conservative interventions had treatment benefit in carpal tunnel syndrome. Surgical treatment has a superior benefit, in symptoms and function, at six and twelve months. Patient underwent surgical release were two times more likely to have normal nerve conduction studies but also had complication and side effects as well. Given the treatment differential and potential for adverse effects and that conservative interventions benefitted a substantial proportion of patients, current practice of a trial of conservative management with surgical release for severe or persistent symptoms is supported by evidence.
Carpal tunnel syndrome (CTS) is the most common entrapment neuropathy  in America. The prevalence of CTS is from 1% to 3% [2, 3]; with an incidence that peaks in the late 50s . There is a high rate of CTS within certain occupational groups such as meatpackers, poultry processors and automobile assembly workers  which is attributed to job tasks that require intensive manual exertion. In addition, CTS is associated with some systemic conditions, such as rheumatoid arthritis, hypothyroidism, diabetes mellitus, gout, and pregnancy . Both conservative and surgical treatments are used to manage CTS. The non-surgical treatment options include splinting, steroids, activity modification, non-steroidal anti-inflammatory drugs, diuretics, vitamin B-6 and others. However, of the conservative approaches only splinting  and steroids  are supported by high quality evidence.
Surgical release of the carpal tunnel is known to be effective and is typically used for patients who fail to achieve adequate relief with conservative managements and for those with moderate to severe symptoms . Although surgical intervention is considered as the definitive treatment to the CTS, it is not considered a first line of treatment. Conservative intervention may not be curative; but may provide sufficient relief in a proportion of cases. It may also be a patient preference due to concerns about the discomfort, inconvenience or safety of surgery. Conservative management is typically preferred for transient cases of CTS such as those associated with pregnancy or short-term overuse. In other cases conservative management might be used for partial relief of symptoms while awaiting surgery or for diagnostic purposes in determining patient response. Despite, potential variations in indications for one treatment and the associated expectations, there are a substantial proportion of patients for whom conservative management may have provided incomplete relief. These patients require evidence that surgical intervention has is more effective to proceed to surgery.
Systematic reviews provided the best evidence. In 2008, Verdugo et al.  conducted a systematic review comparing surgical and non-surgical treatment for CTS; were able to locate four randomized controlled trials. The objective of this study was to build on this work by adopting boarder inclusion criterion, locating more recent trials that conducting a meta-analysis to synthesize evidence in a more quantitative manner.
A literature search of four databases was conducted in June 2010 for studies addressing effectiveness of surgical or conservative interventions for CTS. The research strategy is list in Additional File 1.
These databases were Cochrane Central Register of Controlled Trials (CENTRAL) (The Cochrane Library Issue 1, 2010), MEDLINE (1980 to June 2010), EMBASE (1980 to June 2010), PEDro (searched in June 2010). Only English language papers were included. Searching of international guidelines, computer searches based on key words, and hand searching for references from previously retrieved articles was used to extend the search strategy.
Research articles were included for review if they met the following criteria:
The study was written in English.
The study was designed as a prospective controlled trial.
The study subjects/patients had a diagnosis of CTS, irrespective of the diagnostic criteria used, etiology of the syndrome, associated pathology, gender and age.
The study compared any surgical with non-surgical intervention.
The surgical treatments include:
Standard open carpal tunnel release (OCTR).
Endoscopic carpal tunnel release (ECTR).
Open carpal tunnel release with additional procedures such as internal neurolysis, epineurotomy or tenosynovectomy.
Open carpal tunnel release using various incision techniques.
The non-surgical treatment includes:
Drugs: oral or local steroids, non-steroidal anti-inflammatory drugs (NSAIDs), diuretics, pyridoxine, etc.
Physical therapy, therapeutic exercises and manipulations. (ultrasound, laser therapy, yoga, and acupuncture, etc).
Research articles were excluded from review if they met the following criteria:
The study investigated the efficacy of two surgical interventions or two non-surgical managements.
The study did not provide data on intervention effectiveness).
The study published before 1970.
Types of outcome measures
The primary outcome measure was patient self-reported functional and symptoms improvement at six months of follow-up. We selected this time point because most studies discussed the post operative status 6 months after the intervention.
Patient self-reported functional and symptoms improvement at three months of follow-up.
Patient self-reported functional and symptoms improvement at twelve months of follow-up.
Improvement of neurophysiological parameters.
Complications and side-effects.
Study authors (QS and JM)) independently performed the study selection, assessment of methodological quality and data abstraction. Structured data extraction forms were used to extract data on the characteristics of individual studies. Information was collected on participants (age, sex, diagnostic criteria used to confirm CTS, severity of symptoms, duration of symptoms, inclusion/exclusion criteria, trial setting, allocation procedure, blinding, number of participants or hands randomized), interventions (description of interventions, treatment length, number and explanation for any drop-outs) and outcome measures (description of measures used, continuous/dichotomous nature). We used the Cohen's (unweighted) kappa to assess the agreement between the two reviewers on study selection.
All the articles were assessed by two reviewers (QS, JM) using Jadad et al. scale  (see Additional File 2) and the Structured Effectiveness Quality Evaluation Scale (SEQES) (see Additional File 3) independently. All the disagreement was solved by consensus discussion.
Jadad et al. scale is used to assess the methodology quality of each study. There are 3 criteria for this scale and total score ranges from 0 to 5. We decided that the study was high quality if the cumulative score was 3 or more. To add additional detail on the quality of studies we also used the SEQES . The scale has 24 items, scored 2, 1, or 0 based on congruence with specific descriptors. In this review, each study was ranked as low, moderate, or high quality based on the cumulative score (/48) using the following metric:
Low quality: scores 1-16
Moderate quality: scores 17-32
High quality: scores 33-48
Statistical analysis was performed using Review Manager (RevMan) version 5.0 . Relative risks (RR) were calculated for dichotomous outcomes and weighted mean differences (WMD) for continuous outcomes. Studies were compared for heterogeneity using the Chi-square statistic (P-value < 0.05 considered statistically significant) and an I2 test ( I2 >50% considered substantial heterogeneity). A fixed- effects model was initially used in this systematic review. A random-effects model was applied if heterogeneity existed. We conducted a priori hypothesis to explain the heterogeneity that might exist between the studies. The potential sources were: difference in populations, severity of the disease, duration of the symptoms, intervention techniques, length of treatment and methodological quality.
Description of included studies
There were 1333 articles identified from the literature research. Based on the abstracts review, only 10 articles potentially met the criteria for inclusion in this systematic review. Of these, 3 were excluded (Additional File 4) during evaluation of the full article based on the previously established exclusion criteria. Thus, seven primary studies [13–19] were included in the systematic review (Table 1). There was good agreement in the selection of trials (Cohen's unweighted kappa = 0.79).
We assessed seven studies including 5 RCTs and 2 controlled trials in this review. Overall, three studies compared surgery with steroid injection[16.17.18], two for surgery versus multi- modality [13, 15] one for splinting  and one for laser . There was homogeneity in entry criteria. The majority of patients enrolled in studies had clinical diagnosis of CTS confirmed by electrodiagnostic studies. Those had severe thenar muscle atrophy were excluded since these cases are not typically considered appropriate for conservative management.
The methodological features of each study are summarized in additional files 5 and 6. Totally four studies [13, 16, 17, 19] rank high quality according to Jadad Scale. Because of lack of appropriate blinding, all the studies were rated as zero at the criteria of "double blinding". In Demirci et al. and Elwakil et al. articles, there were no adequate randomization performed so that both studies got zero in this criteria.
The quality of all studies ranges from 29 to 40/48 using SEQES. There are two high quality studies evaluated the multi-modality (SEQES = 35-39), two high quality studies and one moderate for the steroid (SEQES = 31-38), one high quality study for the splinting (SEQES = 40), one moderate for the laser (SEQES = 29). The common shortcomings of the studies were lack of blinding and inadequate randomization.
All studies concluded that both surgical and non-surgical intervention were beneficial to patients. However, there were no consistent outcome measures among identified studies. Patient self-reported scales, researcher-assessed subjective impairments, muscle strength and electrophysiological studies were commonly used in these studies. Five studies [13, 15, 16, 18, 19] employed patient self-administered functional and symptomatic scale questionnaires to evaluate the effect of the surgery. Although, these questionnaires varied across studies we were able to pool four studies that included scales for disease specific hand function and symptom to conduct a meta-analysis.
Patient self-administered scales
One high quality  and two moderate quality studies [15, 18] compared surgery and steroid injection and/or splinting by assessing outcomes using the Boston Questionnaire . The Boston Questionnaire is a CTS specific tool for patient to self-report the symptom severity (11 items) and functional status (8 items). The over-all score is calculated as the mean of all items which is from 1 to 5. The higher the score is, the worse the symptom or function is. In previous studies [20, 21], the validity and reliability of Boston Questionnaire has been tested. One high quality study  compared surgery and splinting with Carpal Tunnel Syndrome Assessment Questionnaire (CTSAQ) which is similar to the Boston Questionnaire. CTSAQ is modified from the Boston Questionnaire which includes 11 questions for symptom and 9 questions for function. Also, the reliability and responsiveness of CTSAQ has been verified [20, 22]. Gerritsen's study  reported improvement score rather than ending point score in their outcome assessment. We included these results in our meta-analysis because they presented same direction of difference.
Figure 1,2 demonstrates the pooled functional and symptoms score at 6 months of follow up. We found that surgical was superior to the non surgical intervention at six months with the weighted mean difference 0.35 (95% CI 0.22, 0.47) for functional status and 0.43 (95% CI 0.29, 0.57) for symptom severity. Figure 3,4,5,6 presented the postoperative status score at 3 and 12 months. There were no statistically significant difference between surgery and non-surgery with regard to symptoms or functional score at 3 months, but there was a benefit in favor of surgery in terms of function and symptom relief at 12 months ( 0.35, 95% CI 0.15 to 0.55 and 0.37, 95% CI 0.19 to 0.56).
Five studies [14, 15, 17–19]evaluated the electrophysiological improvement 6 months after the intervention. Two high quality trials [17, 19] measured the median motor nerve distal latency while three moderate quality studies [14, 15, 18]assessed the number of normal nerve conduction tests at 6 months follow up. Surgery was found to be superior to conservative management regarding to the improvement of electrophysiological studies (Figure 7 and 8). For median motor nerve distal latency, the pooled effect size was 0.5 (95% CI 0.16, 0.85), indicating 50% more patients got normal distal latency after surgery. The relative risk of having normal nerve conduction tests after treatment was 2.3 (95% CI 1.2, 4.4), also favoring surgery.
Complication and side effect
Six studies [13–17, 19]reported complication and side effect of surgery and medication intervention ( Figure 9). A number of minor adverse effects were reported including: painful or hypertrophic scar, stiffness, swelling or discomfort of the wrist, most of them were resolved spontaneously in few weeks. Some authors  reported all complication regardless the severity while others only declared clinically important adverse events. This results in a large variation across studies in terms of complication rates. Overall, the pooled relative risk indicated a higher rate of complications in the surgical group (RR = 2.03, 95% CI 1.28 to 3.22). The most common complications reported in the surgical group were skin irritation and wound hematoma; while the complication reported with splinting was swelling of the wrist, hand and finger.
Despite, the limitation in the number of randomized controlled trials available in current literature, this systematic review was able to provide evidence that CTS symptoms improved in both interventions.
All the studies reported that both conservative managements (splinting, steroid and laser therapy) and surgery result in clinically significant improvement in symptoms. Some authors [13, 15, 17–19] concluded that surgical decompression produces long-term systematic improvement compared with the non-surgical intervention. We found that the positive impact of conservative management plateaus within 3 months whereas, the clinical effect of surgical intervention up until 12th months after the treatment. The relative advantage of surgery at 6 months (WMD = 0.35) indicated that patient with surgical release had approximately 0.35 points lower functional scores than those receiving conservative intervention.. Although there was a similar trend at 12 months, no further improvement was observed at 12 months of follow-up. Thus, the current treatment approach of providing a conservative management as a front-line treatment in mild to moderate cases before considering surgery is justified.
However, surgery was superior to the non surgical intervention regarding the improvement of electrophysiological study. The relative advantage of surgery (RR = 2.3) indicated that approximately twice as many patients achieve better outcomes with surgery. This is important information for patient who fails conservative management to understand when deciding whether they should consent to surgery.
Prognosis was not addressed in these study trials but others have indicated that patients presenting with higher symptom severity scores and those not responding within the first six weeks are more likely to proceed to surgery following conservative management . Given that the size of the treatment advantage for surgical management is relatively small, and that improvements are noted with both conservative and surgical approaches the evidence does not support proceeding directly to surgery. The presenting symptoms/nerve damage, response/relief after conservative management, comorbid issues and patient circumstances/preferences will determine the optimal decision about surgery. There are potential complications that patients must consider, in particular for surgical management or steroid injection. Given the huge variation of how complications are defined, this systematic review was not well positioned to determine accurate rates of these complications.
Our review indicates substantial heterogeneity in effects between studies. This may have resulted from variations between the studies in terms of intervention techniques, length of treatment, methodological quality, etc. For example, all the patients in splinting group received 6 weeks treatment in Gerritsen study  while patients in Ucan study  used the splinting for 3 months. For this reason future systematic reviews that included larger numbers of studies might be useful to differentiate subgroups who would benefit most from conservative versus surgical management or factors associated with successful treatment in either treatment arm.
Critical appraisal of trials involving surgery, or hands-on interventions within the scope of conservative management have some inherent challenges in blinding that affect their scores on most critical appraisal instruments. While the Jadad scale is commonly used, others have pointed out its lack of reliability and validity with respect to surgery and rehabilitation research [24, 25]. For this reason we used a 24-item structured evaluation instrument  that has been used in other hand surgery/therapy systematic reviews [27, 28]. This instrument also provides extra credit for blinding, but has an intermediary score for cases where blinding is not possible. In addition, because it addresses a variety of aspects of study in addition to blinding there is an opportunity for well-designed surgery trials to be favorably rated despite a lack of blinding.
One limitation of this systematic review is only studies written in English were included, which might introduce a publication bias. However, one recent assessment reported that non-English papers are likely to be of low quality and could result in bias into a review .
We observed a small to moderate incremental benefit in surgical group for patients with carpal tunnel syndrome. However, given that conservative management is effective in relieving symptoms and can circumvent the need for surgery in a certain proportion of cases it remains a justified first line treatment. Therefore, we do not see a need for further trials comparing conservative management versus surgical management but rather a need for better prognostic studies that would identify the characteristics of patients most likely to respond to each type of intervention. This would form a basis for clinical prediction rules and clearer criteria for which patients should be fast tracked to surgery and how long conservative management should be sustained before making decisions about transitioning into a surgical procedure.
This systematic review presents that both surgical and conservative interventions are beneficial in the management of carpal tunnel syndrome. Surgical treatment provides a better outcome up to twelve months in terms of symptoms and restoration of normal nerve conductions test results; but has higher complication risk. Most complications of CTS interventions are mild. Since conservative interventions are beneficial for a substantial proportion of patients and effects plateau within three months the traditional approach to use a trial of conservative management in patients with mild and moderate or transient CTS is supported by evidence.
Practice parameter for carpal tunnel syndrome [summary statement]. Report of the Quality Standards Subcommittee of the American Academy of Neurology. Neurology. 1993, 43 (11): 2406-2409.
Katz JN, Stirrat CR, Larson MG, Fossel AH, Eaton HM, Liang MH: A self-administered hand symptom diagram for the diagnosis and epidemiologic study of carpal tunnel syndrome. J Rheum. 1990, 17 (11): 1495-1498.
Atroshi I, Gummersson C, Johnsson R, Ornstein E, Ranstam J, Rosen I: Prevalence of carpal tunnel syndrome in a general population. JAMA. 1999, 282 (2): 153-158. 10.1001/jama.282.2.153.
Bland JDP, Rudolfer SM: Clinical surveillance of carpal tunnel syndrome in two areas of the United Kingdom, 1991-2001. J Neurol Neurosurg Psychiatry. 2003, 74 (2): 1674-1679. 10.1136/jnnp.74.12.1674.
Bernard B: A Critical Review of Epidemiologic Evidence for Work-Related Musculoskeletal Disorders of the Neck, Upper Extremity, and Low Back. Cincinnati: Centers for Disease Control and Prevention National Institute for Occupational Safety and Health publication. 1997, 97-141.
Spinner RJ, Bachman JW, Amadio PC: The many faces of carpal tunnel syndrome. Mayo Clin Proc. 1989, 64 (7): 829-836.
Verdugo RJ, Salinas RA, Castillo J, Cea JG: Surgical versus non-surgical treatment for carpal tunnel syndrome. Cochrane Database Syst Rev. 2008, CD001552. Review, 4
Marshall S, Tardif G, Ashworth N: Local corticosteroid injection for carpal tunnel syndrome. Cochrane Database Syst Rev. 2007, CD001554, 2
Bland JD: Carpal tunnel syndrome. BMJ. 2007, 335 (7615): 343-346. 10.1136/bmj.39282.623553.AD.
Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ: Assessing the quality of reports of randomized clinical trials: Is blinding necessary?. Control Clin Trials. 1996, 17 (1): 1-12. 10.1016/0197-2456(95)00134-4.
MacDermid JC: An introduction to evidence-based practice for hand therapists. J Hand Ther. 2004, 17 (2): 105-117. 10.1197/j.jht.2004.02.001.
Review Manager (RevMan) [Computer program]. Version 5.0. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration. 2008
Jarvik JG, Comstock BA, Kliot M, Turner JA, Chan L, Heagerty PJ, Hollingworth W, Kerrigan CL, Deyo RA: Surgery versus non-surgical therapy for carpal tunnel syndrome: a randomised parallel-group trial. Lancet. 2009, 374 (9695): 1074-81. 10.1016/S0140-6736(09)61517-8.
Elwakil TF, Elazzazi A, Shokeir H: Treatment of carpal tunnel syndrome by low-level laser versus open carpal tunnel release. Lasers Med Sci. 2007, 22 (4): 265-270. 10.1007/s10103-007-0448-8.
Ucan H, Yagci I, Yilmaz L, Yagmurlu F, Keskin D, Bodur H: Comparison of splinting, splinting plus local steroid injection and open carpal tunnel release outcomes in idiopathic carpal tunnel syndrome. Rheumatol Int. 2006, 27 (1): 45-51. 10.1007/s00296-006-0163-y.
Ly-Pen D, Andreu JL, de Blas G, Sanchez-Olaso A, Millan I: Surgical decompression versus local steroid injection in carpal tunnel syndrome: a one-year, prospective, randomized, open, controlled clinical trial. Arthritis Rheum. 2005, 52 (2): 612-619. 10.1002/art.20767.
Hui AC, Wong S, Leung CH, Tong P, Mok V, Poon D: A randomized controlled trial of surgery vs steroid injection for carpal tunnel syndrome. Neurology. 2005, 64 (12): 2074-2078. 10.1212/01.WNL.0000169017.79374.93.
Demirci S, Kutluhan S, Koyuncuoglu HR, Kerman M, Heybeli N, Akkus S: Comparison of open carpal tunnel release and local steroid treatment outcomes in idiopathic carpal tunnel syndrome. Rheumatol Int. 2002, 22 (1): 33-37. 10.1007/s00296-002-0184-0.
Gerritsen AA, de Vet HC, Scholten RJ, Bertelsmann FW, de Krom MC, Bouter LM: Splinting vs surgery in the treatment of carpal tunnel syndrome: a randomized controlled trial. JAMA. 2002, 288 (10): 1245-1251. 10.1001/jama.288.10.1245.
Levine DW, Simmons BP, Koris MJ, Daltroy LH, Hohl GG, Fossel AH, Katz JN: A self-administered questionnaire for the assessment of severity of symptoms and functional status in carpal tunnel syndrome. J Bone Joint Surg Am. 1993, 75 (11): 1585-92.
Heybeli N, Özerdemoğlu RA, Aksoy OG, Mumcu EF: Karpal Tünel Sendromu: Cerrahi tedavi izleminde fonksiyonel ve semptomatik skorlama. Acta Orthop Traumatol Turc. 2001, 35: 147-151.
Bessette L, Sangha O, Kuntz KM, Keller RB, Lew RA, Fossel AH, Katz JN: Comparative responsiveness of generic versus disease-specific and weighted versus unweighted health status measures in carpal tunnel syndrome. Med Care. 1998, 36 (4): 491-502. 10.1097/00005650-199804000-00005.
Boyd KU, Gan BS, Ross DC, Richards RS, Roth JH, MacDermid JC: Outcomes in carpal tunnel syndrome: symptom severity, conservative management and progression to surgery. Clin Invest Med. 2005, 28 (5): 254-260.
Bhandari M, Richards RR, Sprague S, Schemitsch EH: Quality in the reporting of randomized trials in surgery: is the Jadad scale reliable?. Control Clin Trials. 2001, 22 (6): 687-688. 10.1016/S0197-2456(01)00147-7.
Bhogal SK, Teasell RW, Foley NC, Speechley MR: The PEDro scale provides a more comprehensive measure of methodological quality than the Jadad scale in stroke rehabilitation literature. J Clin Epidemiol. 2005, 58 (7): 668-673. 10.1016/j.jclinepi.2005.01.002.
Law M, MacDermid JC: Evidence-based Rehabilitation: A Guide to Practice, Second Edition. 2008, Philadelphia, PA: Slack Publishing
Borkholder CD, Hill VA, Fess EE: The efficacy of splinting for lateral epicondylitis: a systematic review. J Hand Ther. 2004, 17 (2): 181-199. 10.1197/j.jht.2004.02.007.
Michlovitz SL, Harris BA, Watkins MP: Therapy interventions for improving joint range of motion: A systematic review. J Hand Ther. 2004, 17 (2): 118-131. 10.1197/j.jht.2004.02.002.
Egger M, Juni P, Bartlett C, Holenstein F, Sterne J: How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess. 2003, 7 (1): 1-76.
This research was supported by New Investigator Award, Canadian Institutes of Health Research
The authors declare that they have no competing interests.
QS Participated in the design of the study, performed the statistical analysis and drafted the manuscript. JM participated in its design and coordination and helped to draft the manuscript. All authors read and approved the final manuscript.
Electronic supplementary material
Additional file 5: Study Quality (Jadad et al. scores) for 7 included articles. summary of Jadad score in included studies (DOC 34 KB)
Additional file 6: Appendix 6 Study Quality (SEQES scores) for 7 included articles. summary of SEQES score in included studies (DOC 55 KB)
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
About this article
Cite this article
Shi, Q., MacDermid, J.C. Is surgical intervention more effective than non-surgical treatment for carpal tunnel syndrome? a systematic review. J Orthop Surg Res 6, 17 (2011). https://doi.org/10.1186/1749-799X-6-17