So, about comments. PubMed, America’s federally maintained database of scientific papers, is piloting a system that will allow scientists to leave online comments on papers. PubMed is huge—it includes 22 million citations in health, biology and chemistry—so its commenting system would encompass a large portion of the world’s research, especially research in health.
Scientists have never had such a central, well-accepted, public place to comment on papers. Should PubMed commenting catch on, it would add another layer of critique and discussion around published research.
Many scientists, including those who initiated the creation of the PubMed pilot, argue that science needs this additional scrutiny. There may be more than a million scientific papers published every year. How can researchers, policymakers and the public know which ones are important or well-done or spurious? A good paper commenting system would mean “flaws can be identified and corrected and so that the most credit is reserved for works that withstand the test of time,” Michael Eisen, a biologist at the University of California, Berkeley, writes in a blog post.
PubMed worried about ‘irrelevant or uninformed comments,’ Tibshirani said.
Post-publication comments contrast with traditional peer review, which occurs before a paper is published. Comments are potentially open to millions of scientists, while even the best peer review includes the opinions of just a few experts. Comments could also stay open for decades, allowing folks the benefit of hindsight in evaluating research.
Will PubMed have the same troubles with trolls that others, including Popular Science, have struggled with? Well, while the National Institutes of Health wants anybody to be able to read PubMed comments, not everyone will be able to write them.
The creators of PubMed Commons, as the commenting system is called, worried about “irrelevant or uninformed comments,” Rob Tibshirani, a Stanford University health policy researcher, said in a statement. Tibshirani led the first group of scientists to try PubMed Commons Beta. The PubMed Commons’ makers ultimately decided only researchers who have had a paper indexed in PubMed will be able to comment.
“Unfortunately, [this policy] would leave out many people who could add valuable input, including many graduate students, patient advocates and science journalists,” Tibshirani said. “I’m a little worried about this restriction.” He says the joining policy may widen in the future.
PubMed Commons’ restrictions probably eliminate many anti-science commenters. But some may still slip through. After all, fraudulent work does get published and indexed in PubMed. In addition, scientists aren’t immune from unscientific beliefs. Certain celebrity doctors are known for championing unproven cures; I’ve endured one or two illogical, racist rants while interviewing scientists as a journalist.
Scientists interested in joining PubMed Commons can find step-by-step directions here. PubMed comments are not currently publicly readable, but the goal is to have them open to any PubMed visitor in the future, Tibshirani said.
Here at Popular Science, we wish PubMed Commons the best of luck.
