Post by Ivan Oransky and Adam Marcus
The unfolding drama of Anil Potti — a Duke researcher who posed as a Rhodes Scholar and appears to have invented key statistical analyses in a study of how breast cancer responds to chemotherapy — has sent ripples of angst through the cancer community. Potti’s antics prompted editors of The Lancet Oncology to issue an “expression of concern” — a Britishism that might be better expressed as “Holy Shit!” — about the validity of a 2007 paper in their journal by Potti and others.
Unlike newspapers, which strive for celerity as much as accuracy, science journals have the luxury of time. Thorough vetting, through editorial boards, peer reviewers and other filters, is the coin of the realm.
And yet mistakes happen. Sometimes these slips are merely technical, requiring nothing more than an erratum notice calling attention to a backwards figure or an incorrect address for reprints. Less often but far more important are the times when the blunders require that an entire article be pulled. For a glossary of the spectrum between erratum and retraction — including expression of concern — see this piece, commissioned by one of us, Ivan, while he was at The Scientist.
Retractions are born of many mothers. Fraud is the most titillating reason, and mercifully the most rare, but when it happens the results can be devastating. Consider the case of Scott Reuben, a prodigiously dishonest anesthesiologist whose fabrications led to the retraction of more than a score of papers and deeply rattled an entire medical specialty. (One of us, Adam, broke that story.)
So why write a blog on retractions?
First, science takes justifiable pride in the fact that it is self-correcting — most of the time. Usually, that just means more or better data, not fraud or mistakes that would require a retraction. But when a retraction is necessary, how long does that self-correction take? The Wakefield retraction, for example, was issued 12 years after the original study, and six years after serious questions had been raised publicly by journalist Andrew Brian Deer. (Thanks to commenter Tutak for letting us know about this error.) Retractions are therefore a window into the scientific process.
Second, retractions are not often well-publicized. Sure, there are the high-profile cases such as Reuben’s and Wakefield’s. But most retractions live in obscurity in Medline and other databases. That means those who funded the retracted research — often taxpayers — aren’t particularly likely to find out about them. Nor are investors always likely to hear about retractions on basic science papers whose findings may have formed the basis for companies into which they pour dollars. So we hope this blog will form an informal repository for the retractions we find, and might even spur the creation of a retraction database such as the one called for here by K.M Korpela.
Third, they’re often the clues to great stories about fraud or other malfeasance, as Adam learned when he chased down the Reuben story. The reverse can also be true. The Cancer Letter’s expose of Potti and his fake Rhodes Scholarship is what led his co-authors to remind The Lancet Oncology of their concerns, and then the editors to issue their expression of concern. And they can even lead to lawsuits for damaged reputations. If highlighting retractions will give journalists more tools to uncover fraud and misuse of funds, we’re happy to help. And if those stories are appropriate for our respective news outlets, you’ll only read about them on Retraction Watch once we’ve covered them there.
Finally, we’re interested in whether journals are consistent. How long do they wait before printing a retraction? What requires one? How much of a public announcement, if any, do they make? Does a journal with a low rate of retractions have a better peer review and editing process, or is it just sweeping more mistakes under the rug?
These are the sorts of things we’ll cover when we write about a particular retraction, and we hope they’ll form the basis of larger discussions of the obligations of journals. The two of us — both with experience covering science and medicine for the consumer as well as trade press — seem to come across these issues often. The experience one of us, Ivan, has had with Embargo Watch suggests that a blog is a great forum for such conversations. While any particular Retraction Watch post will only carry one of our bylines, we will both contribute to all posts.
A few researchers have studied the issues we plan to raise here. For example, a 2008 paper in the Journal of Medical Ethics found that “the rate of retractions remains low but is increasing” and that:
Although retractions are on average occurring sooner after publication than in the past, citation analysis shows that they are not being recognised by subsequent users of the work. Findings suggest that editors and institutional officials are taking more responsibility for correcting the scientific record but that reasons published in the retraction notice are not always reliable. More aggressive means of notification to the scientific community appear to be necessary.
Those sound like great arguments for Retraction Watch.
This sounds excellent and I look forward to the posts. Although when Ivan first mentioned this, I assumed that you would be covering retractions/corrections of science-reporting rather than science itself. The latter is probably more interesting but any thoughts on covering the former too?
Thanks Ed! We will certainly keep our eye on retractions and corrections of science reporting as well. They are more rare, but just as interesting. Our core interest will be retractions of the science itself.
Covering the oopsies of science reporting is more difficult/intensive because, as we all know, there’s not the same level of prominence in disclosing such errors compared to that of primary scientific journals.
One thought, though: smart version-comparing robots could help with such an endeavor. In particular, I’m pondering Scott Rosenberg’s reporting on changes to a story that Politico (yeah, not science but still) tried to sneak in: http://www.wordyard.com/2010/07/21/politico-slate-and-story-versioning/ Seems to me that someone with the right know-how and resources could (easily?) pull this off. Perhaps interface the “hey this story was changed” data with something like MediaBugs.org (if it ever gains mainstream popularity) for crowd-sourced analysis.
I applaud you both for starting this important blog. Hopefully your putting what you find in the public domain will lead journal editors and the scientific community to create the database K.M. Korpela called for. Meanwhile your blog can serve as an informal collection of retractions. Since this is a blog, you may, along the way write suggestions for reforms. Ivan, I am amazed at how you find time for these new ventures, but I sure am glad that you do! Congratulations to you both for thinking of this and for actually doing it. I know your posts will be well read.
Should be fascinating. Librarians are interested, too, in how the retraction shows up in the online version of the journal (is the article still there but marked clearly? is the article gone? is there no marking) and in research databases.
Sorry to be utterly pedantic, but I don’t think ‘expression of concern’ is the least bit a Britishism.
See, for instance, these ‘expressions of editorial concern’ in Science: 2005 DOI: 10.1126/science.1124185 and 2009 DOI: 10.1126/science.1186078
If they’d said ‘a concern about a spot of bother over a bit of a rum article’, I’d be inclined to see the Britishismness more!
Go to Elsevier’s ScienceDirect site and search for “retracted.” Most results are to articles that have been retracted rather than legit articles that use the keyword. The folks at Elsevier are scrupulous about noting who retracted the article, and why. You can also find “retracted” articles from Wiley and Springer.
Hi,
Also check out http://econjwatch.org/ . They do similiar things for economics. In economics papers are not often retracted but “mistakes” are sometimes reported on Econ Journal Watch and the authors are invited to respond.
Cheers
Elliot
This is an interesting piece but please correct the name of the journalist who did sterling investigative work on the Wakefield paper (and has been under huge personal attacks for it by the pro-vaccine lobby) – it’s Brian Deer, not Andrew.
Boy, I betcha Google’s cache attic could easily do the equivalent of Word’s “Compare Versions.” Wouldn’t that be interesting!
interesting thought Dave! moreover, I am a proponent of open source/publication medical papers like the JMIR. When it is open to the public, it will be easier to track fraud
I love the RW blog.
It is such important work. Another case study (where I wrote about “retraction ethics” illustrates why:
Eysenbach G, Kummervold PE
“Is Cybermedicine Killing You?” – The Story of a Cochrane Disaster
J Med Internet Res 2005;7(2):e21
URL: http://www.jmir.org/2005/2/e21/
doi: 10.2196/jmir.7.2.e21
Great blog!
Here’s a recent case to watch:
http://www.nature.com/news/2011/110107/full/news.2011.703.html
Will do — thanks!
This case emphasizes how necessary it is for investigations of fraud claims to be carried out by people completely independent of the institution where the individual works, independent of the granting agencies and government bodies that supported the individual and independent of the journals that published the work.
Only then can we be sure of the “results” of such investigations.
Does anyone know something similar to this exists for chemistry?
As an aside, few retractions can beat “Manuscript withdrawn for scientific reasons” in the ‘being so vague as to tell you nothing category’, see here for an example: http://pubs.acs.org/doi/abs/10.1021/ja904224y
In this case, I can only assume ‘withdrawn for scientific reasons’ translates to ‘they didn’t bother to check a general chemistry textbook before publishing this’, ‘impossible’, and ‘the referee’s have egg on their face for letting this pass’.
Thanks for the comment. We consider chemistry part of our remit, and have covered some chemistry journal retractions. A few here:
http://www.retractionwatch.com/2010/12/27/no-medals-group-that-retracted-jacs-gold-nanoparticle-paper-retracts-silver-one-too/
http://www.retractionwatch.com/2010/12/17/journal-of-the-american-chemical-society-retracts-gold-nanoparticle-paper/
Thanks for the heads-up about the JACS “for scientific reasons” retraction, we’ll look into it. And we always welcome more tips like that.
Note that there have been TWO retractions of this paper, a year apart. And each will cost you 35 USD to read unless you have an institutional subscription to JACS. See http://blogs.ch.cam.ac.uk/pmr/2011/09/03/reader-pays-a-lot-to-read-%E2%80%9Csodium-hydride-as-oxidant-paper%E2%80%9D-you-don%E2%80%99t-need-to-be-a-chemist/
I found this blog while exploring the breadth of plagiarism within academia. I have a unique insight into just how widespread plagiarism is considering I wrote for the custom writing industry for quite a few years and eventually started my own company in this industry. I actually tried to offer my services to several major universities in terms of aiding them in developing strategies to identify certain characteristics of papers that students turned in who had not actually written those papers but who had, in fact, utilized a custom writing service. As I quickly discovered, colleges and universities tend to beat their drums when confronted with a particular instance of plagiarism by one of their professors or students but when given the opportunity to address the systemic pervasiveness of plagiarism they tend to bury their heads in the sand. I have published an expose of this industry that you might find interesting: “Putting America Through College One Paper at a Time: The Outsourcing of Our American Intellect.” It is available on Amazon for a pittance because I have avoided the traditional publishing route. If nothing else, it will give you an idea of just how widespread and influential that the custom writing service has become and how most colleges and universities just ignore it because to address it appropriately would inhibit enrolments, graduation rates and, ultimately, international student fees as well.
sorry, I meant to leave the link to the text in case you were interested in viewing it-I think you need a kindle since it is a kindle publication but there are sites where it can be downloaded as an epub such as smashwords:
http://www.amazon.com/Putting-America-through-College-ebook/dp/B005GGTFMC/ref=sr_1_1?s=books&ie=UTF8&qid=1313945181&sr=1-1
Just found U. Excellent job
Ivan, have you seen Dr. John Krueger’s article in the Office of Research Integrity Newsletter (December 2012) on “what can retractions tell us?” It might be of interest to your readers. Best, Alan Price
http://ori.hhs.gov/images/ddblock/dec_vol21_no1.pdf
Hey Dave,
I am very interested in this question as it pertains to journalism more broadly. It’s part of a broader set of questions about transparency and accountability in journalism.
What my team and I have done is build an app that helps journalists capture their research processes and share them as video.
It’s easier to show you what I mean than tell you, so check this out: https://thebestdisinfectant.com/2021/08/23/sorry-no-77-of-australians-dont-support-a-ubi/
That’s an example of when I used the tool to track my process, capturing the original publication, the discovery of the error, taking the article offline, and the publication of the correction.
What’s amazing about it is that the subject of this story, whose “versioning error” I discovered, is still on good terms with me. I think that’s a product of the fact that she could see what my process was and that gave her a clear picture of my motives and methods, which made it hard to hold a grudge.
The DigitalGrip interview is down, and seems to have been for 4 years now. Here is a link to an archive version at archive.org: https://web.archive.org/web/20181231161719/http://field-notes.digitalgrip.de:80/2012/03/06/my-interview-with-ivan-oransky-at-scio12-the-transcript/