McClelland McNaughton O'Reilly Two Complementary Learning Systems Hippocampus and Neocortex 1995

May 06, 2011 07:54 PM

from Psych Reviews in 1995  see Jay's website.  free and online.  (also  in my Neurosci files)

famous paper:  thousands have cited it;

first page is 419;

TWO systems:  damage to HC disrupts recent memory but leaves remote memory intact.

Memories are first stored (via synaptic changes) in HC  and these changes

support reinstatement of recent memories in neocortex;

neocortex changes a little after each reinstatement...  remote memory is based on

accumulated changes in neocortex;

Neocortex slowly learns to discover the structure of ensembles of experiences (activated in HC).

HC permits rapid learning of new items without disrupting this structure;

reinstatement of new memories in HC interleaves them with others to integrate them in neocortex.

Scoville and Milner 1957 ...  dramatic amnesia due to bilat MTL lesions;

crucial aspec is temporally graded retrograde amnesia.  (the more recent an experience, 

the less apt it is to be retained)  but process of xfer from HC to NeoC can occur over years in humans.

HC's ability to use past info is only in a limited time window.

this slow gradual process (of lessening dependence on the HC) is what is labeled CONSOLIDATION.

is this crucial to learning and memory  (yes!)

consolidation can take up to 15 years...  what's going on?

page 420  (page 2 of the article)...  our models are not detailed models... just illustrative.

ROLE of HC in Learning and Memory page 420

HC System  =  Ammons horn, DG, subiculum,  EntCortex and Perirhin and ParaHC cortex.

1)  patient HM;  despite bilat lesion:  intellect intact  but with profound anterograde amnesia

but INTACT remote mems.

2)  just affects  DECLARATIVE MEMORY (term introduced by NJ Cohen and Squire 1980...

this includes  Endel Tulving's 1983  episodic memories... individ episodes, semantic memories

but also  semantic memories = KNOWLEDGE of words, factual info and encyclopedic mem.

Spatial learning is impaired in HC lesions  (dates all the way back to O'Keefe and Nadel 1978).

detailed discussion of WHAT exactly is impaired in HC lesioned animals.... subj of debate.

3)  some kinds of learning are complely UNAFFECTED by HC lesions.

Squire 1992 called these  nondeclarative or implicit....

classic example from Brenda Milner 1966 is  skill of tracing a figure viewed in a mirror or

reading mirror-reversed print.  or repetition priming tasks;

completeing a word fragment;  reading aloud a pronounceable nonword;

4)  HC lesions give rise to temporally graded retrograde amnesia.

Page 422  One Account of the Organization of Memory in the Brain

many points in common with other accts beginning with Marr 1971.

Complementary Learning Systems;  both use synaptic connex;

one uses special mem system involving HC.

NEOCORTICAL SYSTEM

They consider perirhinal and parahip Cortices to be an overlap zone btwn Neocortex and HC system.

Consider reading:  neocortex provides contextually appropriate retrieval of patterns of repres

specific propositions, events etc....  content of the target pattern and everything associated must be

able to serve as retrieval cues.

Patterns are elicited by propagation via synapses among the neurons involved.  Knowledge that underlies

the neocortex is stored in these connex.   The KNOWLEDGE is assumed to be embedded in the very

neural circuits that carry out the tasks that use the info.  (middle of col 1 on page 423)

(top of col 2)   accumulated changes provide the basis for correct performance in tasks that depend

on the content... provide the basis of acquired cognitive skills.

col 2 p 423   HC Memory System

The repres of an experience in the neocortex consists of a widely distrib pattern of neural activity.

Each experience will give rise to only SMALL ADAPTIVE changes that are genl NOT sufficient to allow

rapid learning of arbitrary associate conjunctions that provide the substrate for explicit recall of the

contents of specific episodes and for other HC-dependent tasks.

We assume that performance in such tasks DEPENDS on SUBSTANTIAL CHANGES to

strengths of connections in the HC System....

HC and Neo are bidirectionally connected   Neo = neocortex.  patterns go both ways.

We do NOT assume that HC get a direct copy of the pattern in Neo.

Instead, the Neo repres is re-represented in a compressed format

over a much smaller number of neurons in HC.

McNaughton 1989 referred to this compressed pattern

as a "summary sketch" of the current Neo repr.

Such compression can occur without loss of essential info if there is redundancy in the neo rep.

analogy to computer compression schemes:   get v high levels of compression if lots of redundancy.

ANNs = artificial neural networks  commonly used to do pattern compression and decompression

going from input layer to <<higher>>  repres layers and back.   (can use intermediate layers)

see FIGURE 2 by Larry Squire et al (and David Amaral) 1989.   great figure

Reciprocal connections to and from ParaHC gyrus and perirhinal cortex  TO  

Cingulate Cortex ,Insula,  Orbitofrontal cortex, and STG = superior tempr gyrus.

Within the HC itself, an event or experience is repres by a sparse pattern of activity in which

the indiv neurons repres specific combs or conjunctions of elements of the event.

Plastic changes to the synapses in the HC increase the probab. that

a fragment of the pattern will elicit the entire pattern and these synaptic changes in the HC will tend to make this pattern an attractor  (ie a pattern toward which neighboring patterns or incomplete versions will converge);

    During recall, if a part of the pattern representing the episode arises again in the Neo, this will

be translated into a part of the pattern corresponding to the previous event in the HC.

If the (elicited) pattern in the HC is suffic close, then it may lead the HC to settle into the attractor,

thereby filling in the missing aspects of the memory trace.

The return pathways from HC to Neo, together with preexisting intracortical connex, then

reverse the translation carried out by the forward connex,  thereby completeing the Neo

reinstatement of the event pattern and enabling overt responses.

page 424  Reinstatement and Consolidation of HC Memories in Neo.

Reinstaement of HC mems may occur for task-relevant situations, and also

off-line in active rehearsal, reminiscence, and sleep.

Reinstatement of HC mems gives rise to reinstaement in Neo.

this leads gradually to an independence of the mems from the HC (ie they get xfer to Neo).

HC can be viewed as a TEACHER of the Neo.

The same consolidation process applies to the development of the NEO for

SEMANTIC, ENCYCLOPEDIC,a d EPISODIC mem tasks.

SEMANTIC MEMORY = categoires and concepts

ENCYC  TASKS require specific factual info

EPISODIC taks require  info abt specific events or episodes in which the individual was

an observer or participant.

In our view, there is NO SPECIAL DISTINCTION among these 3.

They all depend on initial plastic changes in HC synapses, but the Knowledge

underlying the tasks is eventually xfer to Neo via gradual accumulation of small changes.

Consider the event  Neil Armstrong saying "That's one small step for (a) man"....

having heard it on tv while at a birthday party.

If the accumualted changes to the Neo preserve the common aspects of the reinstated event,

then the S. might be able to do the encyclopedic task of recalling what Armstrong said.

p 425  If time and date is included repeatedly, then you'd learn that too;

if not, then you would not.

Same thing applie sto learning semantic info:  that giraffes have long necks or to learn

a category label of a set of items.  (See eg  McClelland and Rumelhart 1985) to see a

system capable of semantic and catgory learning by gradual accum of small changes.

Evidence and Comment

Neocortical processing and learning:  basic notion is that info proc. takes place via

propagation of neural impulses thru synatic connex....  100 years of evidence.

and amidst  a large number of interconnected brain areas.

The notion that nigher level cog proc is mediated by plastic changes goes back to 

James 1890 and Hebb 1949.

AlthoughNeo synapses exhibit experience-dependent plasticity, the bulk of our understanding of

synaptic plasticity comes from studies of the HC.

HC involvement in some forms of memory  425, col 2

must have  reciproc pathways btwn HC and Neo.

The entorhin Cortex is the final convergence zone for Neo inputs to HC and is the

main structure mediating return projection from HC to Neo.

The EntorC = entorhinal cortex is quite small realtive to the combined Neo =>

highe degree of data compression.

As in Figure 2:  Btwn Neo and EntorC and v v s are the ParahipC and PerirhinC,

which may be intermediate layers in a sophisticated compression-decompression operation.

HC uses reps that are highly specifc to a particular CONJUNCTION or combination of inputs

that are active in particular experiences.  These occur in HC itslef.

(bob:  consider that perhaps it just stores or makes note of perplexity... ie the exceptions to 

the expected patterns known in Neo).

example:  rat exploring an 8 arm radial maze...   HC exhibits highly selective PLACE CELLS.

these neurons define CONJUNCTION of cue that define situations rather than simply places.

page 426

Figure 3:  CA3 coding is very sparse...  very selective as opposed to EntorC

figure 3 shows EntorC  neuron fires as rat goes into every branch of maze whereas

CA3 neuron only fires for one particular arm of the maze.

Mechanism in HC neurons is  associative LTP  = long term potentiation.

(studied since 1973).  

LTP depends on near-synchronous inputs by a large # of convergent fibers.

Exhibited in neurons that go from EntorC to DG = dentate gyrus and to CA3 in HC

as well as synapses that connec.  CA3 back to itself and to CA1.

This form of LTP depends critically on NMDA recepts... activated by the

CONJUNCTION of presynap neuroxmtr releast AND  postsynap depolarizaiton.

Agents that block NMDA receptor prevent LTP and interfere with spatial learning.

Extent of MTL lesion is quite signific... just destroying HC produces milder deficit than

do lesions involving ParaHC gyrus etc.  they speculate that

a lot of intermediate processing may occur in ParaHC gyrus,,,  intermeidate in level and

intermediate in temporal rate  (HC may be v fast...  NeoC is v slow and areas surrounding

HC may be intermediate).

Reinstatement of HC mems:  little evidence that HC reactivation causes reinstatement in NC

but there is evidence of reinstatement in HC itself of recent experiences.

SLOW WAVE sleep  :  during SWS  get  SHARP WAVES in  HC  (Buzaki 1989).

HC SWR (sharp waves) are brif high freq bursts that ast abt 100 ms.

page 427

these SWRs in HC provide optimal conditions for synapt plasticity downstream.

Buzaki showed 1989 that SWRs arise in CA3 and go to CA1 and to output layers of EntroC

and then widely to NC..

SUMMARY  427  this is a provisional account... a framework for thinking...

Our work on ANNs = artificial neural Nets or connectionist models adheres to models of NeoC

with a separate HC for rapid learning.

broadly compatible with lots of data;

Key Questions About the Organization of Memory in the Brain

1)  Why is an HC system necessary?

why not just make changes directly in the NeoC?  (and eliminate the middle man).

2)  Why does incorporation of new material in NeoC take such a long time?

Why not immediately, shortly after initl storage in the HC?

page 435  3 Principles of Connectionist Learning

Attempts to learn new info repidly can lead to catastrophic interference.

So,  1) why is an HC system necessary?

A:  HC is needed to provide a medium (a buffer) for initl storage of events

to avoid interference with the knowledge already in the Neo.

2)  Why does incorp of new material in theNeo take so long?

it takes a long time to allow new knowledge to be INTERLERAVED with

ongoing exposure to exemplars of the existing knolwege (to avoid interference)

page 436:  our claim is that experience can give rise to

gradual discovery of structure thru interleaved learning but not

thru focused learning  and that gradual discovery lies at the heart of cognitive, linguistic and perceptual devel.

What is Strucutre  p 436

Strucutre means systematic relationship within or btwn events, so that novel events can be

efficiently represetned and responded to.

sparrow is similar to robin and canary... so once they've been learned, can just plug in sparrow.

pronunciation of English words:  save, wave, cave.... so u know how to pronounce mave.

Visual structures:  neighboring point have similar depth, orientation, reflectance and such

sets of points define surfaces of objects.... discontinuities occur at edges.

So,  surfaces and edges constit structure... so effic to repres objects by surfaces and edges.

Finally  (col 2 page 436)  consider big  matrices of correlated elements... can just store correlations as is

done in PCA.

Why Discovering Structure Depends on Slow Learning...

1)  learning means being exposed to a set of experiences:

each experience represents a sample from an environment which is a 

distribution of population of experiences 

2)  goal of learning is to derive a parametrized characteriz of the environment that generated

the samples rather than just to store the samples.

3) what is stored is not the examples but just the parametrized characterization...

each new example tweaks the overall characterization (that's the only residue of the example).

we might call this  stochastic, on-line, parameter updating procedures but we call them

page 437:  stochastic learning ...  important that it be slow and gradual to

accurately capture the whole population of training examples.

The basic argument is that when each experience represents a single, stochastic sample from

a population,  is it necessary to aggregate over many samples to obtain a decent estimate

of the population statistic.

Gradient descent procedures...adjust each parameter in proportion to the derivative of

the performance measure with respect to that parameter (holding the others constant).

438  interleaved learning is better than massed learning at acquiring the charac of

the entire population (not just the most recent);

example  PINT (pei nt)  vs  VINT...  by storing PINT along with VINT can use

the reularities of  P, N, T  even though the I part is irregular.

Example:  John F. Kennedy assassination...  understanind it depends on know. of

presidents, motorcades, rifles, spies.  One's understanding of these things

informs and even pervades one's memory of the Kennedy assass.

Integrating ensembles such as this provides the substance of semantic, episodic, and encyclopedic memory.

Shared aspects of the new experience and what's already in NeoC are most easily incorporated.

The idiosyncratic pieces take more time.

Decay of HC traces over time is influential.

If the HC trace decays rapidly in comparison with the rate of consolidation, then much of the

idiosyncratic content of indiv episodes and events will not be consolidated.

So,  arbitrary and idiosyncratic stuff tends to be lost.

SKIP page 439 thru 448   detailed, mathy elaboration...  go to

page 449  GENERAL DISCUSSION

entire point of departure is based on retrograde amnesia and pts like HM (Scoville and Milner 1957).

elaborated by David Marr 1971 and strongly emphasize in Larry Squires work over 20 years.

their proposal builds on ideas by David Marr.  He viewed consolidation as a process of

sorting experiences into categories.

The NeoC is optimized for the discovery of the shared strucutre of events and experiences.

Marr also originated the idea that the HC records new experiences that are then replayed back to the Neoc.

overnight to provide data for category formation.

pursued by several neurophys:  Buzsaki  and by McNaughton...

Buzsaki 1989 elaborate idea that the HC SWRs are the basis of reinstatement.

One central idea is that the HC is the TEACHER of the Neoc.

some theorists (col 2)  have said that the HC directs the choice of reps in the NeoC.

this is different that our diea which is simply that the HC provides training trials to the NeoC.

Wickelgren 1979 suggested that the HC is needed to assign distinct cortical reps to

novel conjunctions of the inputs, so the the NeoC can learn these separately.

or that 1994  the HC may increase salience of neocort reps of cue conjunctions, facilitating

learning of cojunctions.

DiCarlo 1002 and Gluck and Myers 1993  assumed that the HC provides error signlas

that direct the Neo in formation of cue conjunctions or combos.

In summary  page 450   3 different possible roles suggested for HC by various researchers.

1)  aiding the Neo in selecting a repres a t the time of storage

2)  providing a crucial form of repr not available to the Neoc

3)HC plays a time-limited role in the formation of NeoC reps.

Other points of comparison

HC conjunctive coding, spatial rep and the temporal extended role of the HC during consolidation.

learning cue conjunction and spatial learning may be v similar.

451  Explicit and declarative memory

    We have adopted the point of view that rapid formation of novel, conjunctive associations

crucially depends on an intact HC. (includes explicit/ declarative but poss others)

    Flexible use of memory

Cohen and Eichenbaum 1993 suggest that HC specilizes in repres of recent mems that supports flexible use.

this is similar to the distinction btwn mems VS habits  introduced by Mishkin 1984.

Cohen and O'Reilly  suggested that the role of HC is to provide for

rapid autoassociative storage of arbitrary contents of partic events and episodes...

alowing for their reconstruction via reinstatment.

The other parts of the system  (451 top of col 2)  eg PFC can influence HC recall by providing cues to

the HC.

Reference vs Working Memory = WM;

Proposal by Olton, Becker, Handelmann 1979 that HC is nec. for WM = recent info, currently relevant)

but not refrence memory...  is similar to our proposal.

Binding:  Proposal:  HC provides mechanism = mech that BINDS together diverse aspects

of the cortical rep of spec episode or event.

See this in Wickelgren 1979; Squire 1984 etc.

Teyler and Discenna 1986 suggest that HC does not store the mem itself but only pointers

to the diverse location in NeoC where mem is stored.

It may seem that our view is difft, but not really.

Our proposal does NOT require that a full copy of the NeoC pattern be xfer to HC;

rather we assume that HC uses a compressed repr and simply needs to encode enough

info to reconstruct the pattern.

Teyler and Discenna 1986 agree:  they replay their addresses (pointers) with our compressed repr.

Compression is carried out by connex leading TO the HC, resulting in a reduced repr

in the EntorC,  the gateway to the HC.   This reduced repr is what is stored in the HC.

When this repr is retrieved later, return connex from EntorC to NeoC as well as connex within Neoc

participate in reinstatement of the NeoC pattern.

This proposal  page 452 shares with Teyler and Discene 1986 the idea that much of the info

for reconstruction is not stored in the HC  (but rather in the NeoC).

Teyler and Discenna 1986 suggest that the actual content of the HC is stored within local circuits

in the NeoC at the time of learning.  The HC only binds these local patterns together

so they can be reactivated together later.  The plastic changes in local circuits of the Neo constitute,

on this view, the extra info needed to turn the addresses stored in HC into a NeoC memory trace.

This aspect of T and D contrasts with our proposal: we say fast learning is in HC;

slow learning is in NeoC;  T and D think that there is some fast learning in NeoC;

It is possible to combine these views:

On specific poss is that  superficial layers of Neo are part of the fast-learning system with

the HC itslef as the convergence zone (Damasio 1989).

eg there are higher density of NMDA receptors in layers 2 and 3 of Neo than in deep layers

some suggestion that superfic layers use a sparser repr.  

Also,  it is the superfic layers of the NeoC that xchange bidirectional connex with HC.

Aligns (page 452 col 2 top) with proposal that HC may contribute to the assignment of the NeoC repr

at the time of initial learning.

There is an alternative to accepting the idea trhat substantial changes are made to neocortex at

the time of initl mem formation.

The idea is that knowledge for encoding/decoding is laready present in the connex to/and from HC to NeoC.

It these connex were part of the slow-learning NeoC system, their weights would capture the

the structure already characteristic of the ensembles of events previously experienced

enabling successfull compression/decompr of new patterns that exhibit the same structure.

Rapid changes in these bidirec pathways would interfere with th reinstatement of older HC mems.

The proposal that bidirect connex btwn HC and NeoC exploit existing structure in ensemble of experiences

makes a prediction:  that  HC memory will be far superior for material conforming to the structural constraints

than for total random patterns.  Indded this is one of the most pervasive findings in memory lit eg

Bartlett 1932 on Remembering...  memory is far better learned when it conforms to familiar structures.

page 453 (last page of article)  PREDICTION (of future)

autoassoc future events with cues from present or past.

Conclusion

Our work expands roots in the work of David Marr 1970 and 71 and Larry Squire 1984.

(big contribution is a)  HC teaches Neo  b) HC is the seat of fast learning (retention of

events and episodes) c)  HC then  repeats those episodes so that NeoC may slowly incorporate them.)

This scheme prevents catastrophic interference/ forgetting in the NeoC.  HC contains a highly compressed

abstract of an episode.

REfs:  hundreds:  453 to 457  submitted in April 1994 !!!  

McClelland McNaughton O'Reilly Two Complementary Learning Systems Hippocampus and Neocortex 1995

Attachments