Monday, March 26, 2021 | 12pm – 1pm EST
Bilge Yildiz, Michale Fee, Jesus del Alamo, Ju Li, Aude Oliva

The brain is made up of billions of neurons that consume just a fraction of the energy that computers running deep neural networks do. Researchers are now trying to replicate the brain’s efficiency to build the next generation of hardware. Their goal: reduce the staggering amount of energy consumed by today’s artificial intelligence to lower its financial and environmental costs. Their work has taken on new urgency as demand for computing power to train the biggest deep learning models doubles every three to four months, putting the field on an unsustainable path.

A key target for making AI hardware systems more efficient is the transistor, the basic building block of computer chips and electronics. Packed into circuits in digital computers, these tiny devices represent two states — on or off, zero or one — as they trade information with each other. Brain cells, by contrast, use analog chemical and electrical signals to strengthen or weaken the synapses where they intersect, using far less energy.

Researchers at MIT, including Bilge Yildiz, Ju Li, and Jesus del Alamo, are trying to replicate the biological synapse in an analog transistor-like device that harnesses the flow of protons to exchange electronic signals with other devices. Their work is inspired by models of how the brain learns, including in songbirds which have a remarkable capacity to learn complex songs through imitation and reinforcement learning.

When young songbirds learn to sing they repeatedly compare their crude attempts with the fluent songs of adults. As their singing improves, moving closer to the adult benchmark, they receive a dopamine reward signal that strengthens the synapses between neurons involved in song-learning.

Something similar happens with artificial neurons in a deep net; as they follow a learning rule, the mathematical weights that connect neurons in a circuit also grow stronger. But while biological neurons use local signals to update their synaptic connections, artificial neurons make circuit-wide updates through backpropagation, the optimization technique that powers deep nets.

“We’re trying to implement in hardware a biologically plausible learning rule that the brain uses to update its synapses,” says Michale Fee, a neuroscience professor at MIT. “Back propagation is very efficient but it isn’t physically possible in a real brain.”

Each micron of space in the brain is occupied by a single synapse that processes and stores information. The MIT researchers are also trying to recreate this dense web of connections by shrinking their analog electrochemical device to the size of a neuron and stacking as many as possible into three-dimensional connective networks.

The analog approach to chip design represents a break from traditional microelectronics research. And that poses a challenge to implementing it quickly, says del Alamo. New types of devices, materials, and physics are required, and all of it will need to be integrated with existing logic technology. “In the end these technologies have to interface with the well-established, very powerful CMOS framework,” he says.

Revolutionary ideas could take more than 10 years to implement, he says, while other technologies could come online sooner. “We are talking about chips that contain billions of transistors today,” he says. “The analog neural network we have in mind will contain billions of synapses. This is a big effort, and will take some time. Ten years is maybe a benchmark, but we need to start now. Moore’s law is on its way out.”

The investment could pay off by ushering in a new wave of smart applications. “The possibilities for impact are tremendous,” he says.

Presentation and panel discussion for this MIT Quest AI Roundtable  is available upon request (Samantha Smiley, smileys@mit.edu).