Spaun FAQ
<p>Dec 04, 2012 08:29 PM
 
FAQ  by Chris's guys...
 
Xuan:  points in high d space... now extend that to 512 d space.
 
Terry:  projects like IBM Synapse and Blue Brain are def working in that direction but
genl doing one partic component in great detail...  rather than less detail and connecting them all
together.
 
Hardware setup:  
Xuan:   core sim code is in Java.
 
model itself is in Python... cuz its so much easier to write.
 
to sim SPAUN... we use an in-house GPU server and a supercomputer in Ontario...
Sharcnet.
 
Terry:  core res software is just a smile java applic... at nengo.ca
can easily be run by anybody anywhere.
 
can run the model on pretty much any hardwqare.
we have a CUDA version for GPUs...  FPGA version
can upload to SPINNAKER...
 
Terry:  working direclty with KWABENA B
using this model to do brain machine interafce for presothet limbs
see  PDF.
 
whole bunch of pojrects:   whether its Kwabena or
Spinnake  *ARM procs in to one paral sys)
 
TRAVIS... how to get into the field...  perfect...
PYTHON  really nice syntax... look up simple  RL algos
millions of tutorials.
 
TRAVIS:  recom  Chris Eliasmith book on NEF....
recom   Kandel   Memory book 2007.
 
Trevor... looks into Coursera....
take an expt neurosci paper and try to model it!!
 
(Bob:  work with Kwabena on this)
 
Trevor...  we welcome code contribs... start with tutorials and go from there.
 
What's next:
 
TRAVIS:  more learning...
hierarchical  RL and doing 
novel tasks using prev learned solns... then adding learned solns to its repertoire!!!!
 
prof at UWaterloo  wants to incorp robotics into our models... robot eyes and arm
controlled by spiking neuron models built in Nengo!
 
Terry:  getting in more linguistics...
 
goal is to be able to describe a new task to the model and have it do that.
right now its hard-coded to do tasks
ie we SET the connex btwn cortex and BG.
 
Xuan...  make a system to naviage a maze with only
visual input from a screen and motor ouput to a mouse.
 
Opinion on Singularity>  Human level AI
 
Terry:  we are trying to figure out HUMAN INTEL rather than AI in genl.
but tryin to figure out algo behind human intel will def help make human level AI.
 
big part of what comes out of our work is  fiing that  SOME ALGOS are v easy to implemnt in
neurons... other r not.
 
CIRCULAR CONVOLUTION is an EASY OPERATION to implement
but a simple  MAX function is extremely difficult.
 
(my reaction:   have huge community of users trying to implement 
functions in NEF...  what functions can be implemented?)
 
TERRY:  wud be v surprised if human equiv AI within 20 yrs...
 
we've just startged to map  ALGOs... nothing like standard algo...
MUCH CLOSER TO CONTROL THEORY!
interesting challenge to mape those to psych phenom.
 
That said...   actually building complex bio realistic models is the way forward.
 
HOW DID U TRAIN?  Is it it just a giant BACKPROP NET?
 
only the visual system is trained... in order to categorize the handwritten digits;
it groups similar looking digits in a high D vector space.  trained on MNIST.
 
Rest of Spaun is NOT trained...  rather than a gigantic network with is trained...
 
we guess what a part of the brain does and hook it up.... ie hand-crafted.
 
Analogy to how a car works... figure out the necess parts for a working car and put
those together.
 
(bob:  my key issue is COMPOSITIONALITY/ decomposition...
is decompositon throwing away the baby???  must it be  WHOLE?
eg a la Bergson/  Gibson affordances pragnantz... Gestalt?)
 
Terry:  bit of tension btwn ML and Comp Neurosci.
ML  just focused on solving probs rather than how brain does it.
 
So ML  tends to ignore neurosci,  but occas some in ML uses neurosci inspir
to make next big NL algo breakthru...  think of Geoff Hinton
deep belief nets.
I also thik comp neurosci needs to be v familiar with NL so we can USE any algos
that showup there.
 
our model is not started with a blank slate:
we compute what the connex weight shud be.
 
Similarity to  BLUE BRAIN?
 
Trevor:  they're going  bottom up from the details.
We go top-down ... what functions do we want to perform and
build bio plausible tools.
We hope to meet in the middle... and it will be the best collab ever.
 
our key interest is in simulating FUNCTION using bio plausbile neurons/ mechs.
 
Xuan:  Main Q we tried to anser:
can we use neurons to build a brain to to do these things?
 
try to understand each part of the car and what function it has  and construct your own verison.
 
INDIV NEURON MODELS:
 
LIF = leaking integr and fire.
 
Travis:  can use more complex neural models:  want to commun with programs like Neuron
but LIF neurons capture 95% of features... and they are fast to sim.
 
Terry:  we worry abt  neuroxmtr  reabsorp rate:  2 ms to 200 ms
we use only one neuron type:  stnd  LIF neuron.
 
Ethics:   we beleive in open source and public... so negative uses can be id'D.
what is it abt people that lets repressive states take them over...
 
JEFF HAKINS
 
Terry  I've fiddled with HTM models...
terry says " my instinct is that that sort of proc is only one of many types in the brain!
 
our goal is to understand brain... not build brain-like AI in genl.
 
we work at the lvel of neurons;
lots of repre power in the distrib reps that neurons user.
 
future models may have to include fine details abt protein and whatnot...
 
Connectome project:
right now what we want is  part A connects to part B but not to part C...
rather than lowe-level details:  neuron 1,534,123  connects to neuron 34, 678,945 with str 0.275  !!!
 
Jeff Krichmar neurorobotics?  our focus is on how the human brain works.
so our compoents must map onto real brain neurons.
our models are sonstrrained to have same organiz as brain.
our models run slower.
 
SYMBOL MANIP NETWORK vs  Connectionist Net????
 
Terry:   look at our paper in Oxford Handbook:   3rd appraoch:
symbol-like manips  while still being completley connectionist!!!
when building models, easier to think abt symbols than abt neural.
 
Terry:  dodges issue of feelings/ consc...
but emotional cetners are older then cortex... may be simpler to model than language.
 
What limits SPAUN?
 
Terry:  lots of brain parts not in spaun.  we need a better theor understanding of what
those parts do,  writhe that as a math algo and drive how neuron can do it...
then add it to the model.
 
also ... lots of adaptability (learning... that we don't understand.
 
that's limiting us more than compute power.
 
NOAM Chomsky article in the Atlantic:
Terry agrees with Noam...  need theoret understanding.
we need to figure out the underlying strucs and algos...
how to repr and manip structured info.
 
The core past of how Spaun works is a proof-of-concept of whot neurons can
store and manip structured info...   has been v difficult for neural and statist methods.
 
BINDIGNG PROBLEM?
main thing was getting it to run and give vaguely realistic results.
got a good match for serial recall... forgeting items in the middle.
 
spiking patterns when guessing a number..
capre to rats when sguessing which lever to press...
 
Binding prob... big part of our resch.
 
binding prob:  how to keep multiple things straight;
how to repr  8 in pos 1, 4 in pos 2, tc.
is it done by neurla oscillations...
 
that's not what we do...
we use  VSA  Vector Symbolic Architectures...
how to combine pattern to get new patterns that u can decompress to get the originals.
we use CIRCULAR CONVOLUTION...  easy to do in neurons.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 </p>