Stewart Eliasmith.Compositionality Bio Plausible Models OXFORD 2012
<p>Dec 04, 2012 01:55 PM  readable!
 
for Oxford Handbook of Compositionality...   in press.
 
1st author  Terry Stewart.
 
complexities of cognition:  fodor and pylyshyn 88  and Jackendoff 02;
 
<b>Non-Classical Architectures =  VSA   Vector Symbolic Architectures</b>;  (big section)
 
repr their atomic parts via a numberical vector.  but
what values are allowed;  how are vectors combined and how are orig values extracted.
 
We focus on  the VSA called  HRR;  holographic reduced rrepres.
 
Important feature of NEF = Neural Engr Framework:
methods for generating connection weights and representations continue to be applicable
<b>no matter how detailed the underlying models of single neurons is.</b>
<b>Can be applied to  LIF = leaky integrate-and-fire neurons,</b>
<b>adaptive LIF neurons, and even  highly complex compartmental models</b>
<b>that require supercomputers to simulate the firing pattern of a single neuron</b>.
Can add detail  or can simplify to get models requireing less compute power..
once it works, can create more detailed neural model to get
precise predictions abt firing patterns and repres accuracy...
 
Dale's principleL  in real brains... positive (excitatory) and negavite (inhibitory) weights
use difft xmtrs in distinct neurons.  Parisien et al handls that.
 
Being able to <b>incorporate whatever biological detail is deemed relevant </b>for understanding the system
allows the NEF to be a flexible tool for modeling neural systems. Coupling the NEF with the HRR
approach leads to a neural model of compositionality that is consistent with available modern
neuroscientific evidence as to the capabilities and limitations of real physical neurons...
 
<b>Evaluating NEF HRRs</b>
The result of implementing Holographic Reduced Representations using the Neural Engineering
Framework is a detailed, biologically plausible model of compositionality.
The NEF approach to representing values by encoding them in neuron firing patterns is highly
<b>robust both to increased noise and the loss of neurons</b>. For example, in our model of the Wason card
task (Eliasmith, 2005), on average a full third of the neurons could be removed from the HRR
representation before the system became incapable of correctly decoding and applying structured rules.
This is a side effect due to the system beingdesigned to deal with realistic spiking neurons and neural
variability.
 
The NEF provides a direct method for designing neural systems that<b> can transmit HRR representations</b>
<b>from one location in the brain to another,</b> something not considered by the other approaches. Also, the
algebraic manipulations of HRRs can all be implemented by calculating the connection weights
between neural groups. For example, the operation AB=C can be implemented by having a neural
group representing A, a neural group representing B, an intermediate combined representation, and an
final neural group representing C. The total number of neurons required is five times the number of
neurons needed for a single representation. Given our previous calculations, this means about 1.4
million neurons would be needed (9mm...
2
of cortex). However, this same population of neurons can be
used for every binding and unbinding operation,
 
We do not following the standard approach of having particular neural groups
represent particular symbols (i.e. “grandmother cells”)
but instead we claim that symbols correspond
to distributed patterns of activation.
 
although we have assumed random patterns in this paper, we take these patterns
to often include<b> semantic similarity, so that the patterns for cat and dog would be similar in some</b>
<b>important way</b>s.
...
 
 
 
SUMMARY: last couple of pages....
 
neurobio constraints can and should inform theory choice when evaluating theories of
compositionality.
 
is a model implauisble :  requires too many neurons/ implauible connectivity.
 
is high-level  behavior of model comparable to performance limits of
compositionality observed in humans.
 
LISA and neural BBD implement of classical symbol system will be unrealistic.
neural BBD  requires intricate arrangement ... not likely to be robust.
 
tensor product and VSA provide graceful degradation.
HRR=  Holographic Reduced Reps...  best of both worlds:
realistc neural limits and performance.
 
HRRs exhibit compositonal behavior..
 
reprs of the constituents of a stucture are NOT present in the repr of the structure itself.
instad,  noisy versions of these parts must be extracted via algebraic manipulation.
 
New avenues for EVAL.
Numerical comparisons can be made btwn accuracy of system as complexity increase vs people.
The model can also be used t<b>o generate predictions of what sorts of firing</b>
<b>patterns would be observed in neurons performing this sort of task, what connectivity they would have, and even the amount of time it would take to perform structure manipulations</b>. We believe that 
exploiting these sources of evidence will be fruitful for evaluating theories of human compositional
behaviour.
 
 
 
 
 </p>