ICS News Archive

Unlocking the potential of deep learning

At the European Organization for Nuclear Research, better known as CERN, the Large Hadron Collider (LHC) produces one petabyte of data per second. Of this, it only records roughly 6 gigabytes per second of data to be processed—amassing a record of 100 petabytes of data since its first run in 2009. For context, 1 petabyte of data is enough to store the DNA of the entire population of the United States, and then clone them twice over, according to Computer Weekly. A library of all of the world’s texts, in all of the world’s languages, would contain nearly 50 petabytes of data. If we work with Computer Weekly’s estimates, 100 petabytes of data would contain the entire human memories of 80,000 people. That’s the amount of data CERN scientists are working with.

And that’s where new techniques in artificial intelligence and machine learning—especially deep learning—become very useful, saysPierre Baldi, Chancellor’s Professor of Computer Science and director of the Institute for Genomics & Bioinformatics. In November, Baldi traveled to CERN with his graduate student Peter Sadowski to lecture on deep learning at a groundbreaking Data Science @ LHC 2015 Workshop, forging links between particle physics and machine learning.

Deep learning gets its name from the many layers used in processing high-level abstractions. As Baldi explains, “Think about vision, for instance, we don’t recognize images immediately. We detect edges or contours; it’s a process that occurs in many stages. It’s a deep process that requires extracting features and combining them together. Deep learning is the idea that you can train this whole stack of processes together using data.”

Baldi and his collaborator, UCI’s Associate Professor of Astronomy and Physics Daniel Whiteson, have shown that deep learning can be used to improve our ability to detect exotic particles like the Higgs boson, and potentially dark matter. The universe is 80 percent dark matter, and physicists have observed its effects on gravity. But because dark matter doesn’t interact with light, we don’t know what it is, exactly. The particle collision experiments attempt to produce dark matter within the LHC—though dark matter’s existence, if produced by the LHC, could only be inferred through energy and momentum lost after a collision. With this level of obscurity, dark knowledge can improve deep-learning processes and therefore aid in the analysis of dark matter data.

Deep learning is also capable of creating “dark knowledge,” a form of knowledge that is not explicitly present in the training data. Consider a vision system trained to recognize a finite number of object categories, each one associated with a different output. “When a car is presented to the trained system, in the output, you’re getting all kinds of numbers. There will be a large number corresponding to the car category, because the system is well-trained to recognize the image of a car,” Baldi explains. “But it will also have some value for a truck, because a truck is somewhat similar to a car. The intensity for a vegetable, however, will be very low, because a vegetable has nothing to do with a car. So, dark knowledge is the information contained in the relative size of these numbers that goes beyond the fact of whether there is a car or not in the image. It tells you something about the degree of similarity between the categories.”

Dark knowledge can be used to better train shallow learning systems—smaller and faster, but less accurate systems like those on your smartphone—fittingly tying together deep learning, dark knowledge and dark matter.

While the link between deep learning and particle physics is picking up steam, there remain concerns. “There is always some level of anxiety when a new method comes into a field, but deep learning comes with additional anxieties because people view it as a ‘black box method,’” Baldi says. “It may be very good at recognizing the signature of Higgs bosons, but you don’t know how it does this, because the system learns from data through an opaque process. People find this a little unsettling.”

These groundbreaking new techniques upend principles physicists have relied on for decades, but there’s an additional challenge to implementing deep learning on a wide scale. “Deep learning requires a lot of computing resources to process such large amounts of data and to train deep-learning systems. It’s very intensive from a computational standpoint,” Baldi says. Naturally, funding such huge enterprises can be difficult.

Still, applying deep-learning techniques to particle physics experiments represents only the tip of the iceberg in the field’s useful applications. While deep learning has mainly been utilized in engineering—examples include Facebook’s face recognition technology, or Google’s use of deep learning to recognize speech and natural language—the field could be useful in all natural sciences. In chemistry, for example, it is being used to predict the outcome of reactions. In biology, it is being used to predict protein structures, or the effect of mutations.

Ultimately, Baldi says, “Deep learning is the greatest tool we have today to detect faint signals in data. The potential impact is great.”

Online Resources:

• Data Science @ LHC 2015 Workshop (PDFs and recordings)

• Pierre Baldi’s lecture on “Deep Learning and its Applications in Natural Science at the workshop (PDF and recording)

• “Artificial intelligence called in to tackle LHC data deluge” article in Nature