New findings about how the brain interprets sensory information may have applications for treating brain disorders and designing artificial intelligence.
Imagine you’re sitting on a train. You look out the window and see another train on an adjacent track that appears to be moving. But, has your train stopped while the other train is moving, or are you moving while the other train is stopped?
The same sensory experience—viewing a train—can yield two very different perceptions, leading you to feel either a sensation of yourself in motion or a sensation of being stationary while an object moves around you.
Human brains are constantly faced with such ambiguous sensory inputs. In order to resolve the ambiguity and correctly perceive the world, our brains employ a process known as causal inference.
Causal inference is a key to learning, reasoning, and decision making, but researchers currently know little about the neurons involved in the process.
In a new paper published in the journal eLife, researchers at the University of Rochester, including Greg DeAngelis, the George Eastman Professor of Brain and Cognitive Sciences, and his colleagues at Sungkyunkwan University and New York University, describe a novel neural mechanism involved in causal inference that helps the brain detect object motion during self-motion.
The research, funded in part by the National Institutes of Health and the NIH National Eye Institute, offers new insights into how the brain interprets sensory information and may have applications in designing artificial intelligence devices and developing treatments and therapies to treat brain disorders.
“While much has been learned previously about how the brain processes visual motion, most laboratory studies of neurons have ignored the complexities introduced by self-motion,” DeAngelis says. “Under natural conditions, identifying how objects move in the world is much more challenging for the brain.”
Now imagine a still, crouching lion waiting to spot prey; it is easy for the lion to spot a moving gazelle. Just like the still lion, when an observer is stationary, it is easy for her to detect when objects move in the world, because motion in the world directly maps to motion on the retina. However, when the observer is also moving, her eyes are taking in motion everywhere on her retina as she moves relative to objects in the scene. This causes a complex pattern of motion that makes it more difficult for the brain to detect when an object is moving in the world and when it is stationary; in this case, the brain has to distinguish between image motion that results from the observer herself versus image motion of other objects around the self.
The researchers discovered a type of neuron in the brain that has a particular combination of response properties, which makes the neuron well-suited to contribute to the task of distinguishing between self-motion and the motion of other objects.
“Although the brain probably uses multiple tricks to solve this problem, this new mechanism has the advantage that it can be performed in parallel at each local region of the visual field, and thus may be faster to implement than more global processes,” DeAngelis says. “This mechanism might also be applicable to autonomous vehicles, which also need to rapidly detect moving objects.”
Major NIH award spurs findings on causal inference
In 2020, a team of Rochester researchers, including DeAngelis and Ralf Haefner, an assistant professor of brain and cognitive sciences—as well as others from Harvard Medical School, Rice University, and the University of Washington—received a $12.2 million grant award from the National Institutes of Health to study how the brain uses causal inference to distinguish self-motion from object motion.
The five-year award is part of the NIH’s Brain Research through Advancing Innovative Neurotechnologies (BRAIN) initiative. The insights generated by the award may have important applications in developing treatments and therapies for neural disorders such as autism and schizophrenia, as well as inspire advances in artificial intelligence.
“This NIH BRAIN Initiative Award is the biggest research award in the history of the Department Brain and Cognitive Sciences,” said Duje Tadin, professor and chair of the department at Rochester, in 2020. “It aims to solve the key question of how our brains interpret the information collected by our senses. This research builds on a longstanding strength of BCS of using computational methods to understand both behavior and underlying neural mechanisms.”
Unraveling a complicated circuit of neurons
Causal inference involves a complicated circuit of neurons and other sensory mechanisms that are not widely understood, DeAngelis says, because “sensory perception works so well most of the time, so we take for granted how difficult of a computational problem it is.”
In actuality, sensory signals are noisy and incomplete. Additionally, there are many possible events that could happen in the world that would produce similar patterns of sensory input.
Consider a spot of light that moves across the retina of the eye. The same visual input could be the result of a variety of situations: it could be caused by an object that moves in the world while the viewer remains stationary, such as a person standing still at a window and observing a moving ambulance with a flashing light; it could be caused by a moving observer viewing a stationary object, such as a runner noticing a lamppost from a distance; or it could be caused by many different combinations of object motion, self-motion, and depth.
The brain has a difficult problem to solve: it must infer what most likely caused the specific pattern of sensory signals that it received. It can then draw conclusions about the situation and plan appropriate actions in response.
Building on these latest results and using data science, lab experiments, computer models, and cognitive theory, DeAngelis, Haefner, and their colleagues will continue working to pinpoint single neurons and groups of neurons that are involved in the process. Their goal is to identify how the brain generates a consistent view of reality through interactions between the parts of the brain that process sensory stimuli and the parts of the brain that make decisions and plan actions.
Developing therapies and artificial intelligence
Recognizing how the brain uses causal inference to separate self-motion from object motion may help in designing artificial intelligence and autopilot devices.
“Understanding how the brain infers self-motion and object motion might provide inspiration for improving existing algorithms for autopilot devices on planes and self-driving cars,” Haefner says. For example, a plane’s circuitry must take into account the plane’s self-motion in the air while also avoiding other moving planes appearing around it.
The research may additionally have important applications in developing treatments and therapies for neural disorders such as autism and schizophrenia, conditions in which casual inference is thought to be impaired.
“While the project is basic science focused on understanding the fundamental mechanisms of causal inference, this knowledge should eventually be applicable to the treatment of these disorders,” DeAngelis says.
Read more
Neurons can shift how they process information about motionRochester research indicates some neurons may be more adept than previously thought in helping you perceive the motion of objects while you move through the world.
Small eye movements are critical for 20/20 vision
Rochester researchers show that small eye movements humans aren’t even aware of making play a large role in visual acuity.
Why can we see moving objects against their backgrounds?
Research on how humans adjust to perceive objects could lead to new training programs for elderly adults and patients with conditions such as schizophrenia.
