Telling Good Studies from Bad

An informed patient is in a much better position to partner with his or her physician to achieve optimal health. However, medical news can be misleading or hard to understand, and a little knowledge can indeed prove to be a dangerous thing. Even doctors can get caught up in promising preliminary studies and jump to false conclusions. So what can you do? What follows is a primer on interpreting medical news.

• Randomized controlled study. This is the gold standard of medical research. It means that researchers took a group of people and randomly gave some of them a therapy (a medication or prescription for a lifestyle change, for example) and gave the others (the control group) a placebo and compared the two results.
• Placebo. This is a fake treatment. If a controlled study is trying to determine whether a medication works, researchers will give the control group a fake pill so that subjects don’t know if they’re receiving the real thing. This is important because people’s minds can influence outcomes in important ways. Simply thinking that you are getting treated with something can often make you better.
• Observational study. This is the kind of study where researchers observe people as they live their lives and then draw conclusions. For example, researchers might ask people to write down everything they eat and their daily weight. From that data, researchers would draw conclusions about what kind of diets cause weight loss. In other cases, researchers work retrospectively—asking people to look back at their lives and note their lifestyles or drug treatments and their health problems.Though these kinds of studies can be helpful, they have their flaws. For example, unless researchers in the previous example also observed the patients’ exercise habits and measured their metabolisms, the results could be skewed. With retrospective studies, problems often occur because it’s extremely difficult for researchers to find a comparison group that is the same in every way as the group they’ve chosen to observe. A real-life example of the problems of observational studies is what happened with hormone replacement therapy (HRT). The data suggesting that HRT was good for the heart was based on observational studies. When HRT was put to a randomized controlled test, the old thinking was reversed. How could this be? We now assume that the women in the observational studies who took HRT also had healthier lifestyles that contributed to the fact that they suffered fewer heart problems.
• Preliminary data. Before a company or the government will fund a large, expensive trial (some of them run into the millions of dollars), they want to see preliminary data that support the researchers’ hypothesis. Retrospective studies are done first, because they are cheaper. However, their results may not hold up when the larger prospective study is finally done. This is exactly what happened with hormone replacement therapy. The retrospective studies made HRT look great, but the randomized, prospective study showed no benefit.