This is a story about learning.

As you’re reading this story, you might come across information that’s new to you, like this one: The brain chemical known as dopamine may have more than 300 partners in crime that help us learn.

As you read that last sentence, your neurons sparked electrical shockwaves that triggered long-range signals between cells, carrying the information from the light-sensitive cells in your eyes to knowledge-processing regions of your brain. As the electrical signal passed from one neuron to another, the two cells talked to each other via a specialized connection known as a synapse. To retain this new information as a memory — to learn it — your brain will decide which of these connections to strengthen or weaken, making some of its new wiring permanent and pruning away irrelevant or no-longer-useful circuits.

That’s an oversimplification of learning. How it works at the finest level of detail is mostly a mystery. It’s clear, though, that two factors are important for learning: the strengthening or weakening of new connections between neurons, also called synaptic plasticity, and dopamine, the so-called feel-good chemical that plays a leading role in many cognitive processes. But attempts to use computational models to mimic learning relying on these two factors have so far failed to faithfully recreate the process.

A new computational study from researchers at the Allen Institute posits that neuromodulators, hundreds of different kinds of small molecules in the brain that act on neurons, may be the missing link in theories about how we learn. Their study, which was published on December 16, 2021 in the journal Proceedings of the National Academy of Sciences, could also pave new roads in artificial intelligence.

Dopamine is also a neuromodulator and acts broadly in the brain — many, if not most, of our neurons respond to dopamine. But there are hundreds of other neuromodulators in the mammalian brain, many of which are poorly understood. A study published two years ago, led by Allen Institute neuroscientist Stephen Smith, Ph.D., provided a clue: The proteins that recognize and react to neuromodulators, a certain kind of receptor protein, seem to be highly specific for different neuron types. That is, each kind of neuron in the brain also bears a unique neuromodulator receptor, into which a single kind of neuromodulator fits, like a key into a lock.

The diversity of neuromodulators and the diversity of mammalian neuron types, uncovered through ongoing studies at the Allen Institute and elsewhere, could be linked, the researchers posited. And they wondered if this specialization could be important for our ability to learn as well.

“Why exactly such diversity among types of neurons is useful is quite an open problem,” said Uygar Sümbül, Ph.D., an Allen Institute neuroscientist who led the learning study along with Helena Liu, a University of Washington doctoral student and visiting scientist at the Allen Institute. “Through earlier studies, we figured out that these other neuromodulators are cell-type specific. That was the clue that motivated us to try to put together cell types, neuromodulators and synaptic plasticity.”

Learning and machine learning

To connect those dots, Sümbül and his colleagues turned to what is known as the credit assignment problem. When a series of events precedes an outcome — say, for example, an animal comes across an unfamiliar field full of delicious fruit — the brain needs to quickly piece together the actions that go together with that outcome. Is the fruit associated with the fact that it was raining? Or with the scent that led the animal to a new area? At a smaller scale, millions of synapses in the brain are firing at any given time, and only a handful of these will be relevant to the newly found fruit. The brain needs to strengthen those connections and weaken others to learn where the new food is and remember how to find it again.

In machine learning, credit assignment has to do with nodes in a computer program rather than neurons in the brain. A programmer needs to determine how to assign weights to those nodes so that the program can successfully learn. In computer science, the credit assignment problem is typically solved using a certain kind of algorithm. But biology is not software, and this solution is a mechanism that doesn’t exist in a mammalian brain. Understanding how evolution solved the problem will help us understand our own brains and could enable more efficient machine learning approaches.

While dopamine is known to play a role in credit assignment, this chemical acts globally on the brain — in computational models that attempt to faithfully replicate learning, something seemed to be missing. The researchers wondered if adding local modulation to the models in the form of different brain cell types and their associated neuromodulators might be that missing piece. Liu built a model that included six different types of neurons and neuromodulators, each of which had a different effect on local connections; the model seems to accurately mimic how the brain learns, albeit in a simplified way (we have far more than six different types of neurons).

Next, the scientists want to see their theory tested in the lab — a difficult task that’s only recently become possible with modern brain science techniques such as optogenetics or viral tools that allow scientists to manipulate individual classes of neurons. It’s also possible that their findings could help the artificial intelligence field already, the researchers said.

“Many of the world’s computer scientists are very interested in neuroscience these days,” Smith said, “because they’re looking for what we describe in this paper, some insight as to ways that brains might be solving the credit assignment problem that are more efficient than how engineers do it.”

As for the vast differences in the solutions to learning between those that machine learning experts dreamed up and what evolution sculpted in the brain, Smith isn’t too surprised. Neuromodulators are ancient and universal — this kind of molecular signal even seems to predate the evolution of neurons themselves.

“Maybe the real power of the brain comes from the fact that it wasn’t designed by a human,” Smith said. “These ancient modulatory signals have been there all along. They never went away, because that is part of learning and learning is universal among modern animals. But the genetic machinery for this signaling was kind of hidden by being so cell-type-specific. You only see it if you look at single cells, and that’s what was missing until very recently.”