What’s The Benefit of Knee Surgery?

Ok, it’s true. I just titled it this way to get your attention. Now on to this week’s blog entry…

Over my years in medicine, I’ve often heard people mention a clinical trial in which patients who received ‘sham’ knee surgery experienced the same relief as those who had actually undergone real knee surgery. In some ways, this has grown to ‘urban legend’ status among naturopathic medical students. Despite having heard it mentioned several times, I’d never read the study myself, wasn’t sure of the type of knee surgery involved, and wanted to read more – consider this blog entry as a Snopes report on this study. If you’ve never heard of this study, I recommend you take a look, as it’s the only trial I know of that has utilized a ‘placebo surgery’. I think there’s a lot that can be learned from the study, which was published in the New England Journal of Medicine in 2002.

Here's a basic run-down of what happened. Three groups of people with chronic knee pain and osteoarthritis, approximately 30 in each group, were randomized to either receive an arthroscopic lavage, arthroscopic lavage and debridement, or a placebo surgery. In the placebo group, the patients were put under anesthesia, had the same incisions made as those who were receiving actual surgery, and then had their knees manipulated, as though they were undergoing surgery. In both the short term and long run, those receiving the placebo surgery did just as well as those who received the real surgery, and this applied to not only subjective measures of pain, but also mobility as measured by researchers.

As I said, this has achieved near-urban-legend status in some circles, and I don't think it's quite deserved. Here are my conclusions about the study.

The type of surgery studied matters. When I hear people talk about this study, they rarely say what kind of surgery was performed – it’s usually just ‘knee surgery’. The truth is that the type of surgery studied is vitally important. Though the patients were evaluated radiologically to determine the severity of disease, the type of surgery performed (arthroscopic lavage and debridement) is largely done to reduce patient symptoms, rather than correct a grossly limiting deformity. Were the knee surgery performed to repair a fracture or an ACL tear, I doubt that we would have seen such a drastic response rate from a placebo surgery. Symptoms are at the same time extremely important and slightly tricky – because they are subjective, they can be influenced by factors aside from the physical pathology itself.

What you’re looking for matters. This was not a study to demonstrate the power of placebo, as is sometimes suggested. In fact, this study was done to investigate the mechanism of action and efficacy of arthoscopic knee surgery, and in fact the investigators went to great lengths to see if they had missed any side benefits from the procedure that they had not been looking for. In the world of scientific study, only what is being studied matters, not incidental findings – this study shows that the surgery is not very effective, not that placebo is remarkably effective. If the study had been intended to investigate the placebo effect, it would have been designed differently. Similarly, any additional data points we might pull out of the study can’t be considered valid, because they were not actively studied and so researchers were not controlling for them.

Who you study matters. As the researchers point out, the group of patients who took part in the study was a self-selecting group that may have been more easily influenced by a placebo effect. This may or may not have been the case, but it highlights an important issue in research. When you’re working with a small number of research subjects, you sometimes end up with results that are not reflected in the population as a whole because the research subjects all share an unanticipated trait which is relatively uncommon in the general population.

Don’t distort the results. As I said earlier, this study shows that this surgery is only as effective as a placebo procedure, and that the surgery does not provide any additional clinical benefits that were not initially studied. I’ll go on to talk about placebo in a moment, but as I said earlier, this study was not designed to study the effect of placebo, and so placebo-specific factors were not taken into consideration when designing the study. The researchers note that placebo was more effective than they had anticipated, but even so, we shouldn’t distort the results of this study.

So here’s the conclusion. Patient symptoms are simultaneously extremely important but also extremely plastic. Most docs can relate stories of patients who trucked along seemingly unphased by advanced heart disease or rheumatoid arthritis, and others who suffered greatly from relatively minor complaints. Addressing the person who has a disease, rather than only the disease itself, is an important part of medicine, and one at which naturopathic doctors excel. Even so, this study does not vindicate our profession, nor damn MDs and surgons. The past 15 years has seen a surge in the number of studies into herbs, vitamins and other complementary medicines, and they’ve shown remarkable results. We NDs have literally thousands of studies to draw on when explaining our methods – this should not be one of them.