Must a Paper Trail Be Paper?

There was a time when the only choice scientists needed to make about their lab notebooks was whether to use a blue pen or a black one. But these days, laboratory record-keeping is more complicated. As digitally generated data have proliferated, many electronic alternatives to traditional paper lab notebooks have emerged, varying widely both in price—from free to astronomical—and in how rigorously they adhere to accepted documentation standards. “It’s going in one direction, of course,” says David Wright, director of the U.S. Department of Health and Human Services’ Office of Research Integrity (ORI) in Rockville, Maryland. “With more and more generation of electronic data, the traditional laboratory notebook is going to become obsolete. There are some very sophisticated technologies available. Now it’s a matter of institutions and individual research groups taking them up.”

“When we were using traditional paper notebooks, we were actually losing a lot of information by printing high-resolution digital images and pasting them in a notebook. And there were things we couldn’t even put in lab notebooks, like movies.” —Jean-Michel Ané

Years ago, replication, scientific integrity, and intellectual property concerns led to a well-defined set of lab notebook standards. New technologies, including electronic lab notebooks and large quantities of digital data, have complicated things—but the .

Ladd is aware that various electronic lab notebook (ELN) software packages exist, but she hasn’t felt much need to explore their possibilities. “Part of the reason I use paper notebooks is that’s the way I was trained, and it works,” she says. She also worries that keeping an electronic lab notebook would complicate her workflow. “If you have [your notebook] on the computer, then when you have things like printouts or gel pictures that you would normally just tape into your notebook, then I guess you’d have to scan them and enter them electronically. Having to scan everything seems time-consuming and unnecessary.” Ladd also values being able to draw diagrams and cartoons in her notebook as a way of keeping track of experimental setups and thinks that would be hard to do on a computer screen.

Biochemist Christine Payne of the Georgia Institute of Technology in Atlanta says she has never been tempted to switch to an electronic notebook and isn’t aware of any colleagues who have switched. Payne likes the traditional format, she says, and would hesitate to use any electronic solutions that involve putting data on the Internet because questions of who then owns the intellectual property can become “fuzzy.”

Payne sees paper notebooks as the most durable and convenient way of maintaining information. “It’s easy to pass notebooks back and forth between students if somebody’s taking over a project or is repeating an experiment,” she says. “And if they bring their lab notebooks to meetings, we can easily flip through the notebook together.”

Regaining control

As holdouts from an earlier, paper-driven era, Ladd and Payne are by no means unusual. Reliable figures on lab notebook practices are not available, but it is widely agreed that a large majority of university researchers still use paper. Nonetheless, some scientists have found that the volume of digital data their labs generate necessitates a new approach to record-keeping. “Traditionally, the way that people have managed their information was that they had a lab notebook and they had their minds, and they would remember things,” says University of California, San Diego, neuroscientist Maryann Martone, who is the principal investigator of the Neuroscience Information Framework, a National Institutes of Health-funded collection of laboratory informatics resources for neuroscientists. “They would remember that ‘P756’ or ‘Tomo #7’ referred to an experiment in a particular project.” If they needed to pull up data under a particular project, they could go through their notebooks and recognize those cryptic references.

Increasingly, Martone says, scientists are finding that that sort of dependence on memory has become untenable. “With physical data, we used to have boxes of slides and so on. Now, things are in different computers and are not always available for search.” Some scientists have told her they feel as if they’ve lost control over their own laboratories. “They spend an inordinate amount of time looking for things, so much so that they sometimes decide it’s easier to just redo experiments rather than waste time trying to track down previous results.”

Some of the pressure to streamline laboratory record-keeping has come from scientific journals, which have begun requiring more rigorous documentation. The Royal Society recently began requiring that data associated with papers published in its journals be deposited in an appropriate digital repository. The Journal of Comparative Neurology requires researchers to enter details about their reagents into a publicly accessible database. “All that’s going to start with your lab notebook,” Martone says. “If you’ve got to go look through 60 volumes of your postdoc’s lab notebooks, as opposed to having a beautiful online record, your view of electronic records starts to change.”

Electronic options

Researchers itching to move their notebooks into the electronic realm can choose from many options. John Trigg is the founder of phaseFour Informatics, a London-based consulting company focused on electronic lab notebooks and other laboratory informatics products and services. He says that many academic researchers use homegrown ELNs, adapting general-purpose word-processing software for note-taking, or using collaborative tools such as Evernote or Microsoft OneNote. The benefit of such an approach is that the software is generally inexpensive (or even free), familiar, and—with a small amount of imagination and configuration—easy to deploy. “For a small research group, software selection and implementation can be treated as a small-scale project,” Trigg says. However, he notes, such software leaves much room for uncertainty about security and data integrity, as well as IP, legal, and regulatory compliance. For some researchers, that may not be a major consideration: Some fields have not historically regarded such safeguards as necessary. Compared with industry, Trigg notes, “academia tends to be more relaxed about requirements for the integrity and authenticity of anything that is recorded, though that may be changing a bit now.”

For those who need purpose-built ELN software, many options exist, though some are prohibitively expensive for smaller labs. “Because most of these products have been developed for a pretty well-resourced group of people in pharma and the chemical industry, the prices tend to be pretty big,” says Alan Wolf, who helped coordinate a recent pilot study designed to find an ELN suitable for university-wide adoption at the University of Wisconsin (UW), Madison. For some programs that the university considered testing, Wolf says: “We were looking at price tags that would’ve been well over $1000 per user per year, just for the licensing. In a large lab, that would’ve been a significant chunk of their resources. We knew we’d have to take those off the table. They just weren’t going to work.”

The UW research team selected two products for testing by UW scientists: eCAT, produced by the Scottish company Axiope, and irisnote (formerly CERF). Both products are oriented toward chemists and biologists, but both provide flexible templates that may allow them to appeal to a wider range of scientists, Wolf says. Both programs got high marks from researchers who participated in the trial, but partly because of its lower cost, eCAT came out on top. Wolf says the university is in discussions with Axiope about the specifics of their next version and hopes to make a final decision within the next few months.

Early adopters

Because the goal is to put everything that happens in the lab online, an electronic record is essential to those who practice open-notebook science. In the open-notebook world, the most common tools are .

Developmental biologist Neil Hukriede of the University of Pittsburgh in Pennsylvania switched to irisnote about 5 years ago, mostly as a way to manage digital images. “Each investigator in my lab probably generates thousands, if not tens of thousands, of images per year,” Hukriede says. It was impossible for lab members to paste all those pictures into their notebooks, and there was no way to link their image files, stored on flash drives and computers, to the corresponding lab notebook entries. Irisnote stores the files in a central repository—the lab’s server now contains several hundred thousand images—and allows investigators to link them to corresponding notebook entries, and vice versa.

An added benefit of an ELN, Hukriede says, it that the electronic system makes it easier for researchers to keep track of their old lab notebooks when they leave the lab. “If a postdoc in my lab is leaving, I can generate PDFs of all of her notebooks that she can take,” he says. “If she wants her own digital notebook system, she can take the raw data from irisnote and migrate it to any other Java-based notebook system. And if irisnote ever got bought out or went belly up, we’d still have the ability to migrate the data.”

UW Madison agronomist Jean-Michel Ané is enthusiastic about eCAT, which he purchased for his lab about a year ago after a lengthy search. Until he became a faculty member 8 years ago, Ané says, he had never computed the cost of paper notebooks. When he stopped to tally his expenses, he realized that he was spending more than $1000 a year on paper notebooks for the 10 members of his lab. Hoping to save money and more easily manage the large volume of digital data that his lab produces, Ané searched for an ELN that would be suitable for his molecular biology laboratory. He unearthed many options, but all were out of his price range, were oriented toward chemists, or demanded too much informatics expertise for him to set up on his own. Ané kept tabs on the industry, and when Axiope released eCAT about 2 years ago, he was eager to try it. “It was functionally comparable to the much more expensive commercial software I had tested earlier,” he says. And it was affordable—cheaper than just buying paper lab notebooks every year, in fact.

Ané believes ELNs offer many advantages. One is the ability to record information more thoroughly than is possible with paper lab notebooks. “We keep much more information than before because it’s so easy to copy and paste or to import data,” he says. “When we were using traditional paper notebooks, we were actually losing a lot of information by printing high-resolution digital images and pasting them in a notebook. And there were things we couldn’t even put in lab notebooks, like movies. Now our images go directly onto the server and into the students lab notebooks.” Students can access their notebooks when they’re away from the lab—for example, when using equipment in another department. The ELN also simplifies long-distance collaboration, Ané says.

Now that everyone in his lab uses an ELN, Ané finds it easier to supervise his students’ record-keeping habits. Using a template he set up, students’ record details of each experiment and note any safety concerns. From his own computer, Ané periodically scans ongoing experiments to make sure students are updating their notebooks regularly and that they are including sufficient detail about experiments’ purposes, methods, and results, along with a brief discussion. Once a record has been digitally “signed” by a user, it cannot be changed. The lawyers at the university’s technology transfer office are satisfied that the process meets conventional standards for rigorous laboratory record-keeping, Ané says.

The one disadvantage of using an ELN, Ané says, is that it limits students’ ability to have their protocols on the bench or in the lab’s radiation room, where they can’t bring laptops. Some students cope with this limitation by printing out protocols and bringing them to the bench with them. “We tried for a month to use an iPad to get around that issue, but eCAT is not really optimized for iPads. It’s kind of small for them to read their protocols. But I’m sure that the technology is going to mature. My feeling is that ELNs will be the future.”

Logistical barriers

Peter Herz, CEO of irisnote, says the fundamental barrier for any company aiming to sell an ELN to academic scientists is the fact that compared with desktop or laptop computers, paper notebooks have a negligible footprint. “The real benefits of having an electronic lab notebook are … derived once you have all the information in, because then you’ve got very powerful searching capability, you’ve got very easy collaboration,” he says. “But those benefits are really once everything is in. That’s why, in my view, paper lab notebooks continue to endure, because you have this disconnect.” The value of ELNs comes later, he says, while most of the challenge is upfront.

Herz believes the advent of tablet computers, along with advances in voice and handwriting recognition software and growing acceptance of cloud-based computing, will transform the industry. Like some of its rivals, irisnote is betting heavily on those advances.

It’s an open question whether they will be enough to convince the legions of academics who still use paper notebooks to go digital. “There’s this sort of theory that one day portable devices would come along and solve this problem,” Trigg says. “I’m an iPad enthusiast, but I don’t personally feel comfortable with it in a laboratory environment. I think it’s a great tool for browsing the Web and maybe some light e-mailing, but it’s not a great device for inputting data or having multiple things visible at the same time. As a consequence of that, I don’t see it in the lab. I can see it as something that you use to reference your notebook, for example when you’re sitting in a meeting. But in terms of data inputs, I’m less convinced.”

Martone believes that as digital technologies continue to mature and as norms for transparency in science strengthen, the benefits of electronic notebooks will eventually outweigh the practical barriers. “I think it’s almost a matter of necessity,” she says. “Right now this is very alien to most scientists, but I think the community will come up with flexible solutions that work for them.”

Proprietary software, such as eCAT, irisnote, and their many rivals isn’t the only vision for the future of electronic notebook keeping. A different model has taken hold among some advocates of “open notebook science,” a term that refers to the practice of making one’s entire research record public online. Biologist Karmella Haynes of Arizona State University, Tempe, keeps her lab notebook on , one of many open-source wikis designed to serve as electronic lab notebooks. The software automatically logs changes to notebook entries, creating a publicly accessible history that cannot be tampered with (except, perhaps, by skilled hackers). Haynes requires all members of her lab to use the electronic notebook. “Some people get a little nervous about the open nature of the notebook,” she says. “But in principle, notebooks are not supposed to be secret anyway if you’re practicing true open science.” She also finds the system convenient. “Searching an electronic notebook is so much easier and faster than flipping through pages and trying to decipher messy writing,” she says. The main drawback of the wiki notebook, Haynes says, is that it’s “clunky” to add images. “There are so many intermediate steps. You have to edit the image and then upload it to the site and then link to it. It’s much easier to take a printout from one of the gel imagers or a developed film or a dry gel and just paste it [in a paper notebook].” She hopes that one day soon, someone will develop an app that can take a snapshot and send it directly to a wiki notebook page.

Whatever type of laboratory notebook you choose, the goal is the same: to create an accurate, complete, organized record of your work. Many university legal and tech transfer offices advise that any laboratory notebook, paper or electronic, should include a clear and detailed description of: • The experiment’s purpose • How and when you performed the experiment • What materials you used • What results you obtained • Where related computer files, samples, slides, or photos are stored • Any other information that you or another scientist in your field would need to precisely replicate your experiments Because laboratory notebooks can be used to support intellectual property claims or to defend against allegations of scientific misconduct, all entries should be in immutable form. In a paper notebook, that means using a hardbound notebook with sequentially numbered pages, permanent ink, and dated entries. Errors should be crossed out with a single line, corrections initialed and dated. Electronic records should meet the same high standards of security. Entries should be time stamped and inalterable, and records should be backed up regularly. John Dahlberg, director of ORI’s Division of Investigative Oversight, says his office sees one end of the record-keeping spectrum. In a large number of cases in which there are allegations of scientific misconduct, he says, lab records have been poor or nonexistent. In some cases, the only records are a list of numbers of dubious origin, he says, with no instrument printouts or other direct evidence that the work was done. Other times, he says, lab records consist of Microsoft Word or Excel files whose authenticity can’t be verified. Dahlberg emphasizes another key to good record-keeping: supervision. “Often it’s the case that the lab heads are too busy and they’re not looking at the raw data, so they’re not aware that people in their labs are keeping poor records,” Dahlberg says. “It’s very important that all mentors or lab chiefs make sure that everybody is keeping the lab notebook according to the data retention standards of their institution, and that they’re looking at raw data at regular intervals.”

Science’s Next Wave Policy DebateResources

Engineering Crossroads