Mind Rewind: Brains Run in Reverse
By Ker Than
When faced with a new learning task, our brains replay events in reverse,
much like a video on rewind, a new study suggests.
This type of reverse-replay is also used in artificial intelligence research
to help computers make decisions. The finding could explain why we learn
tasks more easily if we take frequent study breaks: the pauses between
sessions give our brains time to review information.
The finding was detailed in a Feb. 12 online issue of the journal Nature.
Running rats
The researchers measured brain activity in rats as the animals ran back and
forth on a linear track. Specifically, they monitored a brain region called
the hippocampus which is known to be important for memory and navigation in
both rats and in humans.
When the rats completed a lap, they were given a food reward. After eating,
the animals would pause briefly before starting another lap. Outwardly, the
rats didn't seem to be doing much during these rest periods. They would
fidget, groom or remain still. The brain recordings told a different story,
however. During times of rest, a rat's hippocampus was a hotbed of activity.
As the rodents ran up and down the track, hippocampal cells fired in certain
patterns. This sequence of firing repeated when the animals rested, but in
reverse order. The reverse-replays were repeated several times; each replay
took only a few hundred milliseconds.
"In that compressed time, the rat is replaying the entire track from where
it currently is all the way back to the very beginning," said study
team-member David Foster from the Massachusetts Institute of Technology.
"This result suggests that the immediate experience is actually
recapitulated several times. The processing going on outside of the original
experience may be important for learning."
Opening moves
The finding could help explain how rats solve something called the "temporal
credit assignment problem." And because the hippocampus in rats and humans
perform many of the same functions, the current study suggests that our
brains may work in the same way.
The problem, a classic dilemma in decision-making theory, is this: If an
animal has to perform a sequence of actions before it can get a reward, how
does it know which actions were ultimately important and which weren't?
Actions performed right before the reward was obtained are easy to identify
as important, but what about actions performed at the beginning of the
sequence? Which of those were important?
Richard Sutton, a computer scientist at the University of Alberta, Canada
who was not involved in the study, likens the problem to playing backgammon
for the first time.
"How do you evaluate the opening move if you don't know how to play yet?" he
said.
In the fields of computer science and artificial intelligence the temporal
credit assignment problem is solved by having the machines work backward,
replaying events in reverse and assigning more credit to actions near the
end of a sequence than to those at the beginning.
"You know that the final move was the right thing to do, so you can send
that information back through the set of actions that were taken leading up
to the final state," Foster said in a phone interview.
If reverse replay also takes place in humans, it could explain why cramming
hours before a test doesn't typically work. The new finding suggests that
our brains learn best when there are frequent pauses between study sessions;
during these breaks, our brains unconsciously reviews the new information
several times, making it easier to commit to memory when the time comes.
How reverse replay leads to learning
Scientists have long known that the release of the chemical molecule
dopamine is an important part of the brain's reward system. The release of
this neurotransmitter floods us with feelings of joy and motivates us to
perform certain activities.
When this knowledge is paired with the new suggestion that our brains may
replay new experiences in reverse, a possible mechanism for learning
emerges, Foster said.
The researchers hypothesize the existence of a special "value area" of the
brain where dopamine signals and reverse-replay signals are fed become
paired together. If the dopamine signal is one that decays over time,
meaning that it is stronger at the beginning of transmission than at the
end, then the following might happen:
As a reverse replay signal plays out in the brain's value area, it is
associated with the beginning of a strong dopamine signal; as the replay
continues, the dopamine signal becomes weaker. In this scenario, actions
taken near the beginning of a reverse replay event will be more important to
an organism than actions taken later.
Hints in psychology
Sutton said he would not be surprised if reverse replay occurred in animals
as well as machines. If anything, he said, this mechanism had long been
suspected from early psychological experiments, such as Ivan Pavlov's
classical conditioning experiments with dogs.
"Pavlov rang the bell and gave the dog the steak and after a while, just
ringing the bell was rewarding," Sutton told LiveScience. "So somehow it
worked backward from the steak to the bell."
Foster agrees, but added that the current study suggests we make trains of
associations going much further back than previously thought.
"It's taking the animals several seconds to run around, so this replay could
be sending that information back through several stages and rewarding a long
sequence of actions," Foster said. "It's that long sequence that is new."
The current study looked specifically at spatial learning; however, in rats,
and probably in humans too, the hippocampus is involved in other types of
learning as well.
"So [reverse replay] could very well be a mechanism to deal with a broad
variety of information, not just spatial," Foster said.
source:
news.yahoo.com/s/space/20060213/sc_space/mindrewindbrainsruninreverse