 |
| (photo credit) |
Let's take this article as our example (you can click on the red to open it).
At first glance, it looks pretty legitimate - right? The website is the "Children's Medical Safety Research Institute." Can't argue with a title like that. It sounds like it's some sort of fancy, cutting edge collaboration of the best scientific minds around. And I mean it has the word "safety" in it, so obviously this "institute" has only the best of intentions!
 |
| Seriously - does it get more legit-looking than this? |
But if you poke around a little on google, you'll find that all it is is a website of alleged "research" funded by a married couple, neither of whom is any sort of physician or scientist. It closely mimics the name of the fully legit "Children's Medical Research Institute" but don't be fooled - it's not the same. And don't miss the "click to donate" button about two-thirds of the way down the article.
But... the post is based on a real study! See? There's even a cool infograph!
 |
| Cool infograph |
Is it? Hmm. Let's look closer.
Clue #1: Even if you know nothing about determining the legitimacy of an online study, you might be tipped off by the article's claim that the study, a "pilot study of 666 homeschooled children," is the "first of its kind." The article itself states that "Remarkably, not a single published study has ever compared vaccinated kids to unvaccinated kids to see who is healthier years after the shots." Remarkable indeed! D'oh! How on Earth did we not think to do this until just now - we've been debating vaccine safety for decades and not a single scientist has thought to compare vaccinated to unvaccinated children years after their shots?! Silly us. In reality of course, there exists a multitude of such studies, not only out of the U.S. but out of other Western nations as well. Here are just a few (this, this, and this)
But even if you come across a "study" whose conclusions actually do seem legitimate, you can still dig further to determine whether - or to what degree - those conclusions actually are. And in fact, you must do so. Because there is a lot of fake "science" out there on the internet. Here's the link to a sting operation by NPR that found that many online "journals" will actually publish fake science for a fee. And here's an NYT article that discusses the "world of fake academia" and why it exists. How do you dig further? Here are a few ways:
(1) First, google the journal that the study was published in and add the phrase "impact factor." A journal's impact factor is the official measure of the yearly average number of citations, in other journals, to recent articles published in that journal - in other words, it's a measurement of whether or not, and at what frequency, other medical and scientific journals are citing the journal you're curious about. The leading medical journal in the U.S., the NEJM, has an impact factor of 59.558. Bogus journals may have a very low impact factor or, in the case of the "Journal of Translational Sciences" from the article we're examining, no impact factor at all (but be careful - the Journal of Translational *Medicine* is a legitimate journal... now do you see what they're doing?).
(2) Consider the qualifications of the authors of the study. I usually look for studies that have at least one author who holds an M.D. plus a relevant scientific post-graduate degree (e.g., an M.D. and a Ph.D.). If you see a study authored by multiple M.D.s. and Ph.Ds., that study is much more likely to be legitimate than a study with only a few authors whose qualifications are scant or nonexistent. Also, if you see a Ph.D., makes sure it's a relevant one. Remember "Dr. Laura" (Schlessinger)? Her Ph.D. was in physiology, but most of her fans believe she actually has relevant psychological training and education.
(3) Examine not only the "conclusions" section but the actual data. If you look closely at either of the two "studies" home birth proponents rely on to claim that home birth is just as safe as hospital birth (this one and this one), you'll see that in fact, they are relying on the sad reality that most people will only read the "conclusion" section and not actually examine the data in any meaningful way. In fact, when the second such study was first released (by MANA, the Midwives Alliance of North America), HuffPo made the mistake of believing MANA's "conclusions" (this "study" was really just self-reported data MANA was forced to release and had tried to hide), and ended up having to completely change the title of its article covering the study. WOW, right?
(4) Realize that as a laypeople, our examination of any single study will always lack synthesis. Complex medical questions are not often (really - not ever) answered with a single study, and unless we get our own medical degrees and/or our own M.S.'s or Ph.D.'s, our examination of a single study is tunnel-visioned; we often don't even know how to properly be skeptical about it or even how to read its data. So if you find yourself arriving at a conclusion different from the clear consensus of every legitimate health organization in the world (ahem... vaccines), based on your own examination of one study (or a few studies)... you've got cause for pause ;)
(5) Be sure you take a look at the title of the journal. This one just cracks me up! In the post examined above we have a purported study of vaccine safety supposedly published in the "Journal of Translational Science." In fact,"translational" science or medicine has nothing at all to do with vaccine safety and a study comparing vaccinated to unvaccinated children would never appear in a so-titled journal. Translational medicine refers to the efforts to use basic (think cellular) scientific findings to create new diagnostic tools and treatments. Nice try, guys! Now, let's all give a collective sigh of pity for all the people who clicked the "donate" button. Womp womp.
That's my summary folks. Now you, too, can spot fake medical news.
