If Dr Who can't help youngsters who have dyspraxia, who will...

Dr Who?

Dr Who. She is a Time Lord with a friend who has dyspraxia. This isn't about Dr Who. It's about children with poor balance and coordination, who struggle with basic movement skills that we all take for granted.

Dyspraxia/Developmental Coordination Disorder

A good number of children in the UK have dyspraxia: difficulty with movement, balance, and coordination generally. Basically, they have very poor fundamental movement skills.

In the absence of any other diagnosis that would affect motor ability, children with poor movement skills may be diagnosed with Developmental Coordination Disorder (DCD). This affects up to six children in every classroom, and is a diagnosis usually given quite late: children's motor skills are still developing aged six or seven e.g. at the more difficult end of  human motor activity such as balancing on a narrow beam and other agility or gymnastic activities. For this reason, clumsiness might simply be down to late developers and thus diagnosis is delayed. One physiotherapist I know, whose child was diagnosed with DCD, believes that children with DCD are simply well below average. After all, we can't all be Nadia Comeneci.

Poor movement skills have a profound and devastating impact on children's development and life chances. There is a large body of evidence showing how poor movement skills affect children's physical, emotional, social, psychological  and educational development. Researchers have called children with movement difficulties 'onlookers in the playground', and one study found that a large number (80%) of seven year old children with DCD went on to experience drug and alcohol abuse, trouble with the law and mental health difficulties by the age of 22, compared to 11% of the children with typical movement skills. Further evidence also suggests that children’s motor skills have a positive association with physical fitness, body weight and physical activity. Because physical activity tracks across the lifespan, increasing physical activity in childhood can have huge benefits in population health (including mental health) and academic achievement.

Reducing physical inactivity could therefore reduce morbidity but it could also reduce premature deaths and save money. In 2006, research showed that physical inactivity caused 16.9% of premature deaths and costs £8.2 billion per year. However, only 22% of children meet Department of Health recommendations for physical activity, figures which are worsening over the years.  One in five children in Reception and one in three children in Year 6 are obese or overweight. In 2007 obesity cost the economy  £15.8 billion per year, including £4.2 billion in direct costs to the NHS.

In Bradford, a large cohort study (Born in Bradford) has been following over 13,500 children from birth since 2007. During investigations and assessments in schools, we have found that a large proportion of children experience difficulty with their fundamental movement skills. These difficulties have been noted by children's teachers, who have repeatedly asked if there are any exercises that would help these children. For this reason we conducted a systematic review looking for evidence-based activities which could benefit children with poor movement skills. This found nine studies which investigated a total of 16 interventions. Three well-conducted trials in particular found that some physiotherapy activities produced large effect sizes in outcome measures evaluating the children's movement skills.

I have developed these evidence-based activities into a physiotherapy programme designed to be delivered in schools by school staff.

This overcomes the problem for these children of having to wait until formally diagnosed with having a problem before they can be referred for help with physiotherapy or occupational therapy. Besides, there is a sting in the tail for these neglected children because they do not necessarily receive the treatment they so badly need even then. Even before austerity kicked in and reduced numbers of NHS therapists, children with poor movement skills made up 60% of occupational therapy team's waiting lists, and some had to wait up to four years just for an assessment! This is unlikely to have improved in the years of austerity experienced by the NHS since 2007.

Teachers recognise that there is a relationship between poor movement skills, balance and coordination and academic achievement. My qualitative research at a PE teachers and head teachers conference, at meetings with head teachers and from piloting of the evidence-based activities in a school suggests that head teachers and school staff are keen and willing to deliver a programme like this. Indeed, there are already programmes in several schools across the country, but these are not evidence-based or properly evaluated for efficacy.

What are we waiting for!? Let's get it tested !

Unfortunately, despite a number of pilots in schools and some work with teachers and non-governmental organisations, I have been unable to secure funding for an appropriate study to evaluate whether the programme can successfully be delivered in schools by school staff. In the long term, a multi-centre randomised controlled trial (RCT) to evaluate the effectiveness of the physiotherapy programme is necessary, but I first need to conduct a feasibility study to find out whether the programme can be delivered at all by school staff. This feasibility study will also gather essential data needed for properly designing the RCT. 

This research has follwed the Medical Research Council guidelines for developing complex interventions, a process that includes developing the scientific rationale, conducting a systematic review, piloting and then conducting a full trial if justified. So, according to the NIHR definition, a feasibility study finds out 'can this be done?', and does not necessarily need a randomised element. However, I will be evaluating change in children's movement skills, because if there is no change, then a multi-centre randomised trial will be unnecessary. My colleagues are developing a new measure of fundamental motor skills which I hope will be fully tested and published when this research is eventually conducted.

So, what's the problem?

My funding applications have been rejected at the final hurdle, after passing through earlier stages satisfactorily. This has proved extremely frustrating, especially given that I was confident because of the outstanding advice, support and guidance from my awesome mentors, the NIHR's Research Design Service and some senior professors.

The support and positive feedback of senior and highly respected academics gives me confidence in my research, although it does take a battering. One has to be thick-skinned in academia; with reviewers of applications and papers submitted to journals for peer review protected by anonymity, comments and feedback can be pretty brutal, and often just unprofessional or even occasionally stupid. One reviewer (of an NIHR application) wanted to know what  a feasibility study was, even though I had referenced the NIHR's definition and description with every mention of 'feasibility study'.

Well, it is a feasibility study. So, it's at the feasibility study stage.

At one interview, I had to explain the purpose of a control group to a professor.  Another reviewer was of the opinion that it shouldn't be funded in case the children didn't get faster and their hopes were dashed. I hope that reviewer doesn't review any proposals for research into any promising cancer treatments.

In case I look as though I feel hard done by, check out Professor Carol Greider. After receiving news earlier in the day that she was being awarded the Nobel Prize for Medicine, Professor Greider was rejected for a grant by a panel who questioned whether she really knew what she was doing -  in that very area of research for which she was awarded the Nobel Prize! My own professor was once told to get a native English speaker to read through his application before submission.

So the process of applying for funding is frustrating, daunting, painful, time-consuming and occasionally surreal. It takes a thick skin, tenacity and a sense of humour. It takes months to complete some applications. Research is hard, getting funding even harder - but life is hard for children with dyspraxia, and I am determined to see the movement skills programme further developed and tested. It could contribute to better lives for disadvantaged children across the UK and beyond, as well as have significant benefits for population health and the economy.

Virtual clinic for follow up of hip and knee replacement patients

Background to a virtual clinic 

Nearly 200,000 hip and knee total joint replacements were performed in 2016. Because catastrophic failures of joint replacements have poor outcomes for the patient and are difficult to manage for orthopaedic teams, all joint replacement patients are followed up in clinical settings.

National orthopaedic organisations such as the British Orthopaedic Association (which must be envious of the British Hip Society, surely the coolest society in Britain) recommend that outpatient follow-up is one, five or seven years, then every five years for those with implants which have met the Orthopaedic Data Evaluation Panel 10A rating. This places a substantial burden on outpatients, but annual numbers of joint replacements are soaring. If a newly appointed consultant surgeon inherited no patients and started a fresh caseload, by the time he or she had worked for 13 years he or she would see no new patients and would only see 'old' patients in follow up clinics. Yet the failure rate of joint replacements is only 5% and falling. This suggests that a new approach to follow up of joint replacements is called for, one that frees up clinic time but will identify those patients at risk of failure (and needing revision surgery).

A British Journal of Medicine blog highlights beautifully why so many patients need not attend outpatient clinics, and some of the benefits to patients of not having to attend. 

I have been working on the development of a virtual clinic, which is designed to reduce face-to-face outpatient appointments for joint replacement patients by at least 80%. This standardised approach will consist of a patient-reported questionnaire, a standardised radiology report and a clinical algorithm.

Developing the virtual clinic

I developed the virtual clinic using an online Delphi consensus survey. For each of these documents,  anonymised participants with appropriate expertise and experience (joint replacement patients, surgeons, physiotherapists, radiologists and joint replacement practitioners) gave feedback in up to three rounds of the online surveys. With each survey, documents were modified until consensus was achieved.  The patient-reported questionnaire was tested on seven joint replacement patients using a Think Aloud process to capture difficulties with completing the questionnaire, each time undergoing amendments until difficulties appeared to be addressed. 

The final version of the patient-reported questionnaire consists of 13 items, divided into General, Pain, Mobility and Activity sections. There are up to ten items on the radiology report, depending on the element of joint replacement. The algorithm concludes in one of three outcomes: Discharge or standard follow up; See at next available clinic; or Review at surgeon’s discretion (3 – 12 months).

Service evaluation of the virtual clinic

Orthopaedic clinics in five sites across the UK have evaluated use of the virtual clinic in clinical practice. We are still waiting for a large percentage of the data to be returned but 72% of patients reported high satisfaction with the virtual clinic. Almost 80% of patients were given Discharge or standard follow up. 

Findings so far - interim analyses

Out of 317 patients, 120 have returned patient satisfaction questionnaires so far 85% were satisfied with the virtual clinic, with only 2.5% indicating dissatisfaction. Patients were 1.5 times more likely to be satisfied with the virtual clinic if they had mild pain (3/10 or lower).  There was no association between being discharged and patient satisfaction with the virtual clinic.
  
72% of patients said that they preferred the virtual clinic to a face-to-face clinic. Patients were twice as likely to prefer the virtual clinic if they had received a letter or phone call informing them of the virtual clinic outcome, which had not always occurred.

Qualitative feedback from clinicians using the virtual clinic suggests that orthopaedic surgeons save several hours per week of face-to-face clinical appointments. There is a significant administrative burden on other clinical support staff, but this could be relieved with dedicated administrative support (which was organised in some sites). Good support and communication with radiology departments is essential.

Once further data has been received and analysed, I will present more accurate findings from over 500 joint replacement patients. However, it appears that the virtual clinic has great potential and could also be developed and implemented in other clinical areas. 

Expert forum discussion

At the end of October, we are hosting an expert discussion forum in which leading orthopaedic surgeons, radiologists, physiotherapists and arthroplasty practitioners will come together to discuss and vote on the virtual clinic. Joint replacement patients virtual clinic evaluators will be part of the forum, adding their essential contributions. I will update this blog when further feedback and data have been analysed.

Putting the CAT (Critically Appraised Topic) among the pigeons

The Epiphany Club

Research and clinical physiotherapists in Yorkshire have started meeting on a regular basis for mutual support, to share ideas and to improve their capacity for communication and working together.  This is the Epiphany Club, an informal group of physios and other AHPs gathering every few months in a pub near to Leeds Town Centre. 

Click to enlarge

Critically Appraised Topics

One well-received proposal for the Epiphany Club was the development of Critically Appraised Topics (CAT). Keele University School of Health and Rehabilitation has produced a number of very useful musculoskeletal CATs and, over the last fifteen years, their collaboration with clinicians has developed into a hugely beneficial, productive and closely-knit team. A CAT is: 

"a summary of the best available evidence, which answers a clinical question and includes a clinical ‘bottom-line’. A CAT is essentially patient-based, in that it begins with a clinical question generated from a specific patient situation or problem."

This blog will describe the development of the first CAT and its clinical bottom line, generated from the best available evidence. 

Background

No sooner had the CAT idea been put forward than a clinical problem cropped up that was ideal for a CAT. The problem had arisen in a ward-based neuro rehabilitation setting and was causing some anxiety among members of the neuro rehab team. 

After mobilising a patient with an acute partial spinal cord injury, a registrar who had worked at a specialist spinal centre suggested that mobilising patients was inappropriate. He described the current protocol at specialist centres as six weeks' 'bed rest' (or 'Active Bed Based Rehabilitation'). In contrast, the standard procedure on the Leeds Trust rehabilitation unit was early mobilisation when safe to do so (i.e. when medically and surgically stable). But what evidence was there for or against either approach? This then was a clinical question generated by a problem with a specific patient situation, and ideal for a CAT.

Further information: Sir Ludwig Guttmann

There is no doubt that Sir Ludwig Guttmann was a remarkable physican and inspirational human who transformed the quality of care, quality of life and extended the lives of people with spinal cord injuries. The prevailing thought pre-1945 was that rehabilitation was "irrational" and death was inevitable within months, but the principles that he laid down in 1945 were responsible for transforming the outlook and prognosis of people with spinal cord injury. These principles included, for example, the routine turning of patients to prevent skin breakdown. They have remained apparently unchallenged as core practice until at least 1979, but while nobody suggests that turning bedbound patients is not best practice, what is the case for bed rest or against mobilising?

Clinically Appraised Topic (CAT): mobilising acute partial Spinal Cord Injury patients

This Clinically Appraised Topic (Foster et al., 2001) was conducted by Leeds Teaching Hospitals NHS Trust Neurological Rehabilitation team, the Academic Department of Rehabilitation Medicine (University of Leeds) and Leeds Teaching Hospitals Library & Evidence Research Centre, It is described using the the format of the Keele University CATs. 

Acknowledgements

The CAT team would like to thank Kay Stevenson (Consultant Physiotherapist and Clinical Champion, Honorary Lecturer, School of Health and Rehabilitation, Keele University) for her guidance and support leading up to and during the development of this Clinically Appraised Topic.

Specific question 

(click to enlarge)

Clinical bottom line

There was no evidence against mobilising people with partial spinal cord injury (SCI) who are medically and surgically stable less than six weeks after injury. There were no adverse events reported in stable patients with SCI who received early mobilisation.

There was no evidence that bed-based rehabilitation improves outcomes in comparison to early mobilisation in stable patients with partial SCI.

Why is this important?

Once medically and surgically stable, people with acute partial SCI are managed differently in the first six weeks of their rehabilitation depending on the opinion of the professionals caring for them as to best practice. Some centres favour early mobilisation and others support a period (at least six weeks) of active bed-based rehabilitation.

Inclusion criteria

Patients with acute partial Spinal Cord Injury.

Search strategy 

Search terms and databases

(click to enlarge)

Papers found and selection filter

Flow chart 

(click to enlarge)

Summary and assessment of quality of papers

Quality of papers assessed using CASP Systematic review checklist(Critical Appraisal Skills Programme, 2018b) and RCT checklist(Critical Appraisal Skills Programme, 2018a)

Click to enlarge

Click to enlarge

Click to enlarge 

Click to enlarge

Click to enlarge

Summary

There is no evidence that early mobilisation is harmful for acute SCI patients who are medically and surgically stable. Studies are of generally poor quality and do not directly answer the CAT question. 

Evidence from one high quality RCT suggests that overground mobility training is better than body-weight supported treadmill training in acute (2 – 26 weeks) partial SCI patients. There was a positive effect of early exercise on muscle tissue within the first 3–6 months post SCI but these were poor quality studies.

There were weak recommendations in one poor quality paper for early rehabilitation in patients with traumatic SCI when they are medically stable and can tolerate treatment.

There is no evidence supporting six weeks of bed rest.

Suggestions for research

This CAT suggests that properly-conducted research is needed to investigate which approach has better outcomes for people with acute partial spinal cord injury. Given numbers of injuries, this would need to be a large multicentre trial exploring multiple outcomes and long term follow ups. 

References

BAGNALL, A. M., JONES, L., DUFFY, S. & RIEMSMA, R. P. 2008. Spinal fixation surgery foracute traumatic spinal cord injury. Cochrane Database Syst Rev, Cd004725.

CRITICAL APPRAISAL SKILLS PROGRAMME. 2018a. CASP Randomised Control Trial Checklist [Online]. Available: https://casp-uk.net/wp-content/uploads/2018/01/CASP-Systematic-Review-Checklist.pdf [Accessed 01 October 2018].

CRITICAL APPRAISAL SKILLS PROGRAMME. 2018b. CASP Systematic Review Checklist [Online]. Available: https://casp-uk.net/wp-content/uploads/2018/01/CASP-Systematic-Review-Checklist.pdf [Accessed 01 October 2018].

FEHLINGS, et al. 2017. A Clinical Practice Guideline for the Management of Patients WithAcute Spinal Cord Injury: Recommendations on the Type and Timing ofRehabilitation. Global Spine Journal, 7, 231S-238S.

FOSTER, N., BARLAS, P., CHESTERTON, L. & WONG, J. 2001. Critically Appraised Topics (CATs). Physiotherapy, 87, 179-190.

MEHRHOLZ, J., KUGLER, J. & POHL, M. 2012. Locomotor training forwalking after spinal cord injury. Cochrane Database Syst Rev, 11, Cd006676.

PANISSET, M. G., GALEA, M. P. & EL-ANSARY, D. 2016. Does earlyexercise attenuate muscle atrophy or bone loss after spinal cord injury? Spinal Cord, 54, 84-92.

RICE, L. A., SMITH, I., KELLEHER, A. R., GREENWALD, K., HOELMER, C. & BONINGER, M. L. 2013. Impact of the clinical practice guideline forpreservation of upper limb function on transfer skills of persons with acutespinal cord injury. Arch Phys Med Rehabil, 94, 1230-46.

TSE, C. M., CHISHOLM, A. E., LAM, T. & ENG, J. J. 2018. Asystematic review of the effectiveness of task-specific rehabilitationinterventions for improving independent sitting and standing function in spinalcord injury. J Spinal Cord Med, 41, 254-266.

WESSELS, M., LUCAS, C., ERIKS, I. & DE GROOT, S. 2010. Bodyweight-supported gait training for restoration of walking in people with anincomplete spinal cord injury: a systematic review. J Rehabil Med, 42, 513-9.

What should allied health professionals do when their patients ask them about 'alternative' medicine?

Introduction

'Alternative' medicine is a huge business. Boiron, who operate globally selling homeopathic products, made a profit in 2015 of €73.9 million. Cupping as an 'alternative' therapy was given a huge boost in publicity during the 2016 Olympics, when athletes were noted to be covered with the large characteristic bruises.

So what? People can spend their money on anything they like. The Daily Mail, Croc shoes, jeans worn so low they reveal that the wearer's gluteal fold, detox diets and other fad diet regimes, Nestle products... facial tattoos?

Health care professionals are expected to be non-judgmental, after all. But we are health care professionals with a duty of care, firstly to our patients and secondly to the health of the nation, for whom we need to apply "the evidence of what we know works to achieve the step-change in the nation’s health that we all seek."


Is it, therefore, the duty of health care professionals to inform and advise members of the public about the evidence for, and likelihood of, benefits and risks of Complementary and 'Alternative' Medicine (CAM), particularly our patients who ask us or tell us about their use of  CAM?

We have a Bachelor of Science degree; we have expertise in functional anatomy, physiology, and mechanisms of injury, disease and repair. Surely we are qualified to comment, and obliged to do so?


I believe that CAM with no evidence to support its efficacy should not be used in either the NHS, or within physiotherapy practice - NHS or private. But there is evidence suggesting that CAM is widely used within the profession and that our Bachelor of Science degree does not prepare us adequately for appraising the evidence. This is a problem for the health of our patients and the profession itself.

Why should we say anything?

Before I became immersed in research, I worked as a musculoskeletal physiotherapist and as a rotational children's physiotherapist. In both roles, my patients (or parents of children) often asked for advice about the relative merits of 'alternative' medicine but I felt that I wasn't well equipped to answer these questions.

But then one family were keen to try an alternative therapy that promised much greater benefits than those proposed by medical treatment and physiotherapy. The family, desperate for help for their child with a rare and profoundly disabling condition, was low income but willing to do anything - including find and spend the several thousand pounds that this treatment would cost. 

On investigation, I found that the treatment was based on a premise which contradicted the currently established mechanisms of neurological repair; it suggests that improving breathing in disabled children would permit "restorative brain growth" in damaged neural tissue. At extra cost, time spent in an oxygen tank containing 100% oxygen would increase this growth further. There was no research investigating this technique other than an unpublished exploratory study which found no functional benefits (this remains the only research on this technique today). I advised the family that there were unlikely to be benefits from the treatment, based on its proposed mechanism, and that there was no acceptable levels of evidence for any benefits. I also found an anecdote on the Chartered Society of Physiotherapy's interactive forum from a physiotherapist whose caseload included a family already undertaking the same technique. This family, as part of their 'therapy', were told to withdraw from all physiotherapy and associated interventions such as standing frames - the child's physical condition deteriorated to such a severe extent that they became the subject of a Child Protection case.

This anecdote and my experience with the family alarmed me, and I subsequently explored different 'alternative' medicines. I dislike 'alternative' medicine as a description. It isn't an alternative to medicine unless it has been scientifically tested and shown to work - in which case it becomes medicine. Dara O' Briain puts it far better than I ever could, and if you have time, Tim Minchin's song Storm also is worth ten minutes. 

But that's not all. It's scarier than that

Why should health professionals get involved? What's the harm? 

Firstly, a former colleague explained to me that she has no problem with her patients seeking alternative medicine interventions because "they do no harm." 

This assertion is arguable, if not demonstrably misinformed, and I believe that as health care professionals we have a duty of care to inform the public of potential harm, as well as more effective options. For more examples, homeopaths sell prophylactic malaria kits - pure quackery - and chiropractors inform parents who have their babies' spines 'adjusted' that the babies do not then need their vaccinations.

The Chartered Society of Physiotherapy supports quackery

Some physiotherapists are heavily involved in what is as far from evidence-based practice as one can get. There is even a Chartered Society of Physiotherapy-funded Clinical Interest Group for Energy Medicine. 

Energy Medicine was shown to be charlatanism by a remarkable young lady who is the youngest person ever to publish in a peer-reviewed medical journal. Aged just nine years old, Emily Rosa designed and carried out a school science project, an elegant single-blind study on 21 energy medicine (therapeutic touch) 'healers'. The results showed that energy medicine  'healers' can detect an energy field no more than by chance. 

Can you imagine, if you'd never heard of reiki or energy medicine, and you were supervising a student on a musculoskeletal or neurology or respiratory placement, and you asked what they were planning to do with the low back pain/acute stroke/consolidated lung patient. "Oh, I am just going to hold my hands near them and direct healing energy at them."

Energy medicine: literally, 'wishful thinking'. 

It won't surprise many physiotherapists that there is also a Clinical Interest group for acupuncture, a modality which has been subjected to thousands - I think over five thousand - clinical trials. There is still no evidence that it actually works and NICE guidelines now state that acupuncture should not be offered as a treatment option for low back pain. 

NHS physiotherapy teams providing quackery on tax payers' money

According to a thread on the interactive CSP website, an entire team within an NHS Trust is about to start using cupping. The thread suggests that cupping is already in widespread use with physiotherapists.

Cupping is chiropractic nonsense, and this take down and ridicule of cupping AND chiropractic is satisfying for those of us who despair of the gullibility of the press and, sadly, health care professionals who adopt and use these unproven, fanciful therapies. ⚠ Warning: it is probably painful for those of kind and empathic disposition. Even I felt a twinge of compassion for the chiropractor until I remembered he calls himself a doctor, fleeces people, and advises parents not to vaccinate their children. 

They also perform grade 5 manipulations of babies' spines, (at 3 mins 18 seconds - again, not for the faint of heart, it makes me feel physically sick). 

Lack of critical thinking? Lack of applied knowledge? Gullibility?

As with chiropractic, homeopathy, and most forms of 'alternative' medicine, the rationale for craniosacral therapy is nonsense - the light touch (of eight grams) of a practitioner's finger tips manipulating the (fused!) skull bones of the recipient to affect a cure of all sorts of medical problems. A comparison of three physical therapists who used craniosacral therapy to evaluate 'craniosacral rate' (their diagnostic variable) of 12 children all found different rates for each child (their inter rater correlation was -0.02). In other words, each therapist diagnosed a different condition for each child.

In reality, even if I were to bang my head repeatedly against the desk (something I feel like doing when I read about physiotherapists embracing some of these alternative therapies), I would not realign my skull bones a nanometre. Anybody possessing knowledge of human anatomy or a science-based degree should immediately recognise this. Neurosurgeons need drills and saws to move the skull. 

Yet within the Clinical Interest Group for Energy Medicine there is a physiotherapy group promoting the use of craniosacral therapy within the profession. 

I even know of a paediatric physiotherapist who took her child for craniosacral therapy. Apparently, her child's colic (for which there is no known medical intervention) stopped immediately after a few sessions of craniosacral therapy. My own son had colic, and that stopped immediately after a few days of doing nothing.

Critical appraisal of evidence and applied anatomy

For these reasons, I wonder if there is not enough training given to health care professionals to critically appraise evidence. On one thread of the interactive Chartered Society of Physiotherapy's website, set up to argue against the CSP's support of the Energy Medicine Clinical Interest Group and now in its ninth year of debate, one physiotherapist scornfully remarked that science is rubbish anyway - how can a trial be 'controlled' if it is 'randomised'? This suggests a profound lack of training and understanding of basic science methodology. 

Where's that desk...

And despite expertise in advanced and applied anatomy, there are physiotherapists who believe in craniosacral therapy -  that the pressure of a few grams to the skull can regulate the flow of cerebrospinal fluid around the brain. That's an absurd proposition, and proven as such.

Is there any argument for the use of untested therapies?

Elite sports

Leading sports physiotherapist Chris Morgan once discussed with me how elite sport offers time and resources unavailable to many clinicians, such as those working in the NHS. In these cases, he and his medical team can bring to bear perhaps a multitude of evidence-based medical and rehabilitation interventions to maximise recovery. 

But when the athlete is in a race against time to recover from injury for for a one-off event that defines their sporting career - perhaps the European Cup final or the Olympic games - then perhaps every little amount helps, no matter how tiny, to help the athlete recover. The psychological benefits can be as helpful to these athletes as much as any tiny or negligible physical effect. At these times, Chris suggests that “1%ers” could support physiological and psychological recovery of a sportsperson, but stresses they are totally worthless unless they are used only in support of evidence-based treatments.

Against this are the celebrities whose fame seems to lend huge authority to their opinions. 

Jenny McCarthy is a famous proponent of 'alternative' medicine and has been vocal in the argument that autism is caused by vaccines. It isn't, vaccines are harmless and not vaccinating is dangerous. The power of 'celebs' to influence inappropriate health care decisions over and above the advice of experienced scientists and medical professionals is a strange phenomenon of modern society. And it's potentially dangerous.

In the NHS

The potential for harm from untested and unproven 'alternative' therapies has already been mentioned. There are suggestions that Steve Jobs might have survived pancreatic cancer if he had not opted for quackery over the best known treatments of the day.

The NHS is underfunded in so many resources, and physiotherapy teams have been decimated. It is essential that in the limited time physiotherapists have to assess and treat patients they are using evaluations and treatments based on the best evidence.

Physiotherapists in elite sports have more opportunities for implementing 1%ers, for example when the athlete is relaxing, resting from a rehabilitation exercise or receiving a passive treatment. There is also a world of difference between launching an athlete into their career-defining event and raison d'etre, and a postman, painter and decorator, builder etc who returns to a labour-intensive, physically-demanding role with limited paid sick leave.

Private practice

In private practice, physiotherapists are also bound by a professional code. I suggest it is unethical and immoral to take money from (often) desperate patients for unproven and often patently worthless therapies.

What do you think?

Should physiotherapists engage in 'alternative' medicine? 

I could have written pages about various types of nonsense that unethical individuals sell to unwary and ill-prepared members of the public, members of the public who are in despair as they search for a cure to their medical problems. Physiotherapists have specific training and a code of conduct which obligates them to use only interventions with evidence to support them, and certainly not to fleece patients.

As I sit here and write this blog, Horizon (a BBC Science programme) is presenting evidence for healthy eating food fads. This includes a story about a multi-millionaire currently in prison who lied about being a medical doctor and bought his PhD from an online site. He promotes a diet as a cure for cancer. He's made millions of dollars but of course many people have died and the treatment cost them thousands of dollars/pounds. 

What happens when you add Supplements to Complementary and Alternative Medicine?  It reveals SCAM. 

Do physiotherapists have an obligation to advise patients and the public of the evidence for 'alternative' medicine, following objective scientific appraisal of the evidence?

I strongly believe so. But I have suggested that critical thinking and appraisal is beyond some therapists - particularly those who are using 'alternative' medicine in private practice. I have seen use of these therapies defended by the physiotherapist by the cry of 'I have seen the patient get better with my own eyes!'. This supports not the benefits of the 'alternative' medicine but my assertion that these physiotherapists lack critical thinking and objectivity. This has to be addressed.

Finally...

This isn't supposed to be an inflammatory piece, although I accept that it is going to upset some physiotherapists. But I want them to engage in reflective thinking, to consider the evidence and scientific rationale for their 'alternative' therapies, and comment below. But please address my evidence and arguments, and don't resort to attacking me personally. To lighten the mood I leave you with another brilliant comedy take down of 'alternative' medicine. 

I wrote this blog for two reasons, and reiterate that I welcome comments. 

• I wanted to know what physiotherapists thought of the suggestion they have a responsibility to advise the public on the evidence for, and known benefits and risks of, different health interventions; and 

• to voice my disquiet and disappointment at the lack of science-based practice and the failure of the profession and HPCP to address some serious limitations in our professional training and conduct. Is the use of 'alternative' medicine against the principles of evidence-based physiotherapy? What are the HPCP for, if not to regulate this? 

P-values bad: confidence intervals good.

"... the primary product of a research inquiry is one or more measures of effect size, not p values." Jacob Cohen, 1990

P-values are Statistical Hypothesis Inference Testing

When my academic mentor and department head Professor Rory O'Connor asked me if I had ever done any modelling, the only surprise was that nobody had suggested it before. I muttered something about not having the right sort of chin and I was a little short for the catwalk.  He listened to me with his usual fortitude and then sent me to 'Introduction to Modelling', a series of lectures which are part of the MSc in Epidemiology and Biostatistics at my university. 

A common theme in the statistical lectures that I have attended over the last few years has been the widespread misuse, poor reporting and misinterpretation of statistics. The p-value is a particularity relevant and high profile example of this, and it is responsible for false conclusions in at least 30% of research output, more if the study is underpowered. 

The aim of this blog is to explain what the p-value does, why it is unhelpful for judging the outcomes of research and why researchers should report effect sizes and confidence intervals as well. Effect sizes and confidence intervals are easily calculated, and clinicians should look for them when appraising the outcome of research studies. The results of research papers should include the data necessary for calculating them, so I have included the necessary formulas for those occasions where the authors have been forgetful. 

I finish with some examples of papers in which the effect sizes and their confidence intervals contradict the conclusions of the authors.

What does the p-value tell us?

The p-value simply tells us what the chances are of getting a result as large as the one presented in the results if there is no effect of the intervention - if the null hypothesis is true. 

It says nothing about the size of the effect of the intervention or whether the findings can be generalised from the sample to the wider population.

It is difficult to see how p-value has become such a powerful influence on the conclusions drawn from research but it is important that clinicians recognise that interpretation of research findings should include examination of effect sizes and their confidence intervals. And researchers should always include these within their results and draw their conclusions from them. Generally, think of the p-value as 

"something that should be outlawed" ~ Professor Mark Gilthorpe

Effect sizes and Confidence Intervals

Effect size

An effect size is exactly what it says: a quantification of the change in scores of outcome measures or difference between groups. There are different types of effect size e.g. the risk ratio, odds ratio or correlation coefficient (r). They are standardised and can compare changes in the scores of different outcomes. 

An effect size of differences between two experimental groups commonly uses an effect size known as Cohen's d. Cohen's d is based on the groups' mean scores and the standard deviations, scores which are always calculated in analysing results in any case so it is a simple matter to proceed with calculating the effect size. 

And it's a simple calculation:

The pooled standard deviation is:

Because we know the characteristics of standard deviations we know that an effect size of 0.6 indicates that 73% of the control group is now below the average person in the intervention group, up from 50% of course. 

Cohen suggested a rule of thumb for effect sizes: below 0.2, the effect is trivial or non-existent; up to 0.5, the effect is moderate; and above 0.8, the effect size is large. Note that the effect size can exceed 1.

What is a confidence interval and how do we interpret them?

An experiment or research study, and its associated observations, is on one sample drawn from an entire population. A confidence interval gives a range of results which we are 95% confident contains the true population score of interest (e.g. the mean or effect size). 

Why 95% confident? Well, imagine we took 100 samples and conducted the experiment and observations, and calculated means, standard deviations, effect sizes and 95% confidence intervals, on each of them. We would expect five of the confidence intervals not to contain the true population mean/effect size. We would expect each of the other 95 confidence intervals to contain, somewhere in the range of values, the true population score. It is most likely close to the calculated mean/effect size and less likely at the extreme boundaries of the confidence interval. But if the confidence interval crosses the value that indicates no effect, then we must report that the intervention showed no effect. Here is a great interactive visualisation of confidence intervals which demonstrates this.

It's a little more tricky to calculate confidence intervals, but if we've progressed this far it is only lazy and rather self-defeating not to proceed. It's pretty straightforward when you have pulled the relevant figures from the papers.

First you have to calculate the standard error (SE):

The it's putting it all together:

where ES is the effect size calculated earlier, CI is confidence interval and SE is the standard error also calculated earlier. The figure of 1.96 reflects the number of standard errors that would include 95% of the observations. 

An effect size should therefore be reported like this: "the effect size was 0.48 (95% CI: -0.12, 1.08)". 

An example of how effect sizes and confidence intervals change our interpretation of research findings

I will illustrate how calculating and examining effect sizes and confidence intervals can change the interpretation of research findings that have relied only on p-values, using RCTs included in our recently-published systematic review of motor skill interventions for children with developmental coordination disorder. 

Although we set out to include only high quality randomised controlled trials (RCTs), by including only those which scored 7/11 or more on the PEDro scale, we found a number of problems with the studies including that not all of them had calculated effect sizes, and none of them had calculated confidence intervals. 

In order to more effectively evaluate the benefits (or not) of the interventions being investigated by each RCT we calculated the effect sizes and confidence intervals from the data within each paper. This caused us to to interpret the findings of some RCTs quite differently to the authors. 

Does aquatic therapy improve motor skills in children with developmental coordination disorder?

Hillier et al investigated whether aquatic therapy was beneficial for the motor skills of children with developmental coordination disorder, and states in the  abstract:

"Analysis of covariance indicated that at posttest, mean scores on the Movement Assessment Battery were higher for children who received aquatic therapy compared to those on the wait-list (p = 0.057)."

Their conclusion states that "Aquatic therapy was a feasible intervention for children with developmental coordination disorder and may be effective in improving their gross motor skills (my emphasis)". (page 22, but behind a pay wall).

Leaving aside that the ANOVA was statistically non-significant, and so differences in mean scores are meaningless, the abstract implies that the intervention - aquatic therapy - had clinical meaning benefits for children with developmental coordination disorder. Our effect size of 0.66 indicated a  moderate effect - that 75% of the control group would be below the average person in the aquatic group. However, the lower bound of our 95% confidence interval reached below an effect size of 0.2 (to -0.5, in fact), suggesting that there was no effect at all, and certainly disagreeing with their conclusion.

Does table tennis show improve motor skills in children with developmental coordination disorder?

Tsai investigated whether a table-tennis training programme resulted in benefits on motor skills of children with developmental coordination disorder and reported in the abstract a "significant improvement of cognitive and motor functions for the children with DCD". In their conclusion they stated that the children's motor outcomes were "significantly enhanced". 

Our findings contrasted sharply with this conclusion. The ANOVA did show that there were statistically-significant differences between groups across time (p = 0.001) but they did not perform any post hoc testing to evaluate where these differences were, listing only the differences in change scores between groups. These were, for the intervention group, a change of 17.69 (SD 4.26)  improving to 13.38 (SD 2.75) and 18.64 (SD 4.80) improving to 17.57 (SD 4.06). The effect size was 0.95 ( a large effect size), but the lower boundary of the 95% confidence interval  was 0.15. This suggests that in the wider population the effect of table tennis on the motor skills of children with developmental coordination disorder is trivial.

Comparative effectiveness of Pilates and yoga group exercise interventions for chronic mechanical neck pain: quasi-randomised parallel controlled study

Finally, an example from a recent Physiotherapy Journal paper. This paper concludes that reductions in disability (the primary outcome) were significant following Pilates and yoga group exercise interventions. It is great that they also calculated effect sizes - it is far more meaningful than looking at changes in the raw scores, for which they have calculated confidence intervals, but they did not calculate confidence intervals for the effect sizes. 

It is wrong to calculate means, standard deviations and confidence intervals for changes in raw scores of its outcome measure (the Neck Disability Index), as this produces ordinal outcome scores on which arithmetic should not be performed. But I have gone with it, and used their figures to calculate the confidence intervals associated with the effect size. I will also overlook other methodological problems. 


Power calculations indicated 90 participants but only 56 completed the study: this small sample leads to an expectation of a large standard error (SE), and therefore wide confidence intervals. Using the formulas above we find that the reported effect size of 1 (large) does indeed have a wide upper and lower boundary so that the effect size is 1.0 (95% CI: -1.67, 3.67). This suggests that Pilates has no effect on chronic mechanical neck pain as measured by the Neck Disability Index. 

I could have calculated the effect size and confidence intervals using the ordinal outcome scores, if I had access to the full outcome data. For non-parametric data, the effect size is Cliff's d in which one compares each of the scores in one group to each of the scores in the other group, and keeps count of the number of scores in the first group that are greater than the scores in the other group, and the number of scores in the first group that are lower than scores in the other group. This should produce two numbers, which for two groups of 10 total 100 (i.e. 10 x 10). Cliff's d is then the number of scores in group 1 that are greater than scores in group 2 minus the number of scores in group 1 that are lower than group 2, divided by the product of the numbers in each group:

Cliff's d is bounded from -1 to 1, where d = 0 means that there is no effect.

Summing up

There are plenty of other things that can impact on the integrity, interpretation and generalisation of research findings but the p-value has taken on a mythical and invincible might. 

Do not look at the p-value to determine whether the study shows a significant effect of the intervention. Check the effect size - this gives a good indication of the clinical effect -  and look for confidence intervals to evaluate whether this clinical effect could be generalised to the wider population.