Guardians of Science

When she was a medical doctor, Kristen Grace specialized in fertility treatments. Alongside her medical practice, she carried out research in the lab and followed new developments in the field. In her current post as a health scientist administrator (also called a scientist-investigator) at the U.S. Office of Research Integrity (ORI), she does many of the same things: She continues to keep up with findings in the science literature, and she relies on insights borne of her experience to elucidate the problem at hand.

At the ORI, Grace works side-by-side with a handful of other former bench scientists, from various fields, who have traded direct involvement in research for one of science’s most important functions: monitoring and protecting the integrity of science and the scientific literature.

“An assumption that a manipulated image necessarily equals research misconduct is incorrect, since inconsistencies may be due to other factors, including honest error.” —Shara Kabak

From surgeon to sleuth

The move from the OR to the ORI may seem unusual, but Grace has often bucked convention. She waited until her son entered kindergarten to complete a bachelor’s degree, followed that up with a medical degree and, after that, earned a Ph.D. in developmental biology at the State University of New York, Stony Brook. After completing a residency at the Albert Einstein College of Medicine in New York, she left clinical work to pursue research at the National Institutes of Health (NIH). Her work there focused on sperm proteins that can trigger immune responses, leading to infertility.

Then, 4 years ago, as her time at the NIH drew to a close, Grace learned about the opportunity at the ORI. “It was a fortuitous opportunity for me because I wasn’t interested in practicing in the clinic,” she says.

In research labs, she has noticed, ethical decisions are also made against a complex backdrop of real-world complications. When Grace and her colleagues examine misconduct allegations, background issues such as the pressures of funding and the difficulties of running a clinical trial must be weighed. In one recent case, a suburban Chicago cardiologist was found guilty of scientific misconduct for pressuring trainees to change or fake scores on a heart test so that patients would qualify and the clinical trial could go forward. “In this case, the PI [principal investigator] falsified patients’ echocardiograms to make patients appear sicker than they actually were so they could be enrolled in the study.” Because she had been on the researcher’s side, faced with the need to adhere to strict inclusion and exclusion criteria, Grace was able to “tease out details that somebody who hadn’t been on the floor doing research wouldn’t have been able to do,” she says.

What the job is like

When, recently, Grace examined a falsified figure published in a journal, her experiences as a physician and a scientist told her that the doctored image was likely to cause trouble. It could, she realized, jeopardize subsequent research and lead to inappropriate medical treatment. Meanwhile, her experience as a scientist-investigator told her that the alteration was probably just the tip of the iceberg. “Looking at that figure, and knowing the data and the way it has progressed throughout the years, I knew that this data had to have been falsified more extensively over a much longer period of time,” she says. It was, in other words, a serious problem for science.

The structures of science are built to tolerate honest mistakes, but scientists must have confidence that the literature accurately reports the results of actual experiments. That is why fabricating or changing a result is so shocking, says Susan Garfinkel, acting director of the Division of Investigative Oversight at ORI. “Scientists get really enraged when they think that there’s something wrong with the data,” she says. “When you publish a paper, the people reading it have to believe that it’s correct. If there’s any chance that it’s not correct, then you can’t depend on that to support your own research.”

Reports of research misconduct come from various sources. Sometimes, scientists notify Garfinkel’s  office directly when they see questionable data in a journal article. Other cases are sparked by inquiries from journal editors, who noticed something suspicious in a submitted manuscript. No matter where the report comes from, the researcher’s institution—not the ORI—is responsible for carrying out an inquiry to see whether an allegation has merit. If they decide it does, a formal investigation ensues, carried out by the scientist’s institution.

During such an investigation, ORI investigators work with the institution’s research integrity officer (RIO) to guide proceedings and serve as expert consultants to:

  1. offer advice on how to deal with obstructive defendants or aggressive attorneys 

ORI investigators also acquire the data they need to review the case, including a forensic copy of the hard drives related to the allegations. 

When there’s a finding of research misconduct, the case is reviewed by all of the scientists in Garfinkel’s division, as well as by attorneys. The legal standard for a misconduct finding is “preponderance of the evidence,” which means investigators must conclude that misconduct is more likely than not. But because ORI findings are public, Garfinkel says, her office often uses a higher standard. “This includes evidence to prove the data are false, evidence to prove the [accused party] was responsible, and evidence to prove that the falsification is a significant departure from accepted practices,” she says. Investigators must also show that the misconduct had an effect on the research in that field.

Never bored

The scientists currently working in the Division of Investigative Oversight have backgrounds in immunology, genetics, statistics, chemistry, and veterinary science. Without access to such broad knowledge and first-hand experience, the office would find it difficult to make a determination about many specific cases, Garfinkel says.

Garfinkel studied molecular biology as an undergraduate and received a doctorate in genetics from George Washington University. By the time she completed her postdoc, she had grown tired of lab work, so she took an administrative job. “I didn’t feel I was doing anything to contribute to the scientific community, or make a difference,” she says. She also missed the fast pace of lab work. When the opportunity to work at ORI arose in 2003, she took it. “Because every single case that you work on is a little bit different, you just don’t get bored,” she says.

ORI investigators receive no formal training beyond their scientific training—which, however, is essential. An understanding of what lab culture is like is also necessary. “People who have worked in a lab have unique insight into these dynamics,” Garfinkel says, but the scientific perspective is not exactly right for the job: Over time, investigators learn to think more like attorneys, she says. “There’s kind of a steep learning curve when you first come in, to change the way you think about things.” The researchers also must train themselves in forensic analysis techniques for both images and computers, and in how to perform statistical tests that can detect fabricated data.

Better detection methods

Today, scientists with no official investigative role are running their own data-forensics operations. Some even set up blogs with entries pointing to suspicious-looking figures or facts and claiming that they’re fraudulent. Shara Kabak, a scientist-investigator at ORI, says that while her office doesn’t discourage others from looking for wrongdoing, it encourages reporting allegations through appropriate channels: the university or ORI. Proper reporting of allegations, Kabak says, can reduce the chances that crucial evidence will be lost or destroyed. It can also reduce the risk of damaging a researcher’s reputation, she says.  “An assumption that a manipulated image necessarily equals research misconduct is incorrect, since inconsistencies may be due to other factors, including honest error.”

Kabak was introduced to research misconduct while serving on an NIH committee on scientific conduct and ethics. After receiving a doctorate in immunology from the University of Chicago, she moved to Washington in 2007 to become a AAAS (publisher of Science Careers) Science and Technology Policy fellow, working at NIH and following investigations involving NIH intramural scientists. Following a brief stint as a science writer at the National Eye Institute, she accepted the scientist-investigator job at ORI.

ORI’s workload is growing. Last year, the number of cases reported to the office doubled. Kabak says that cases are also becoming more complicated, often involving more than one person. Fortunately, detection methods have also improved, allowing investigators to do a better job of tracking patterns of behavior. “Normally something might have come in and we would have found one thing. Now, with a little bit of digging, it’s easy to find more instances if there’s misconduct there,” she says.

The hub of a big wheel

But the world of research integrity is larger, with investigators at every research institution. With caseloads increasing, that world is likely to get larger still. We’ll present more information about these university-based research-integrity jobs in a future Science Careers article.

Winter Is Coming

In Japan, Seeking More Jobs for ‘Posudoku’