DiCarlo Untangling object recognition video
<p>Nov 26, 2013 09:22 PM</p>
<p>
Jim DiCarlo presents this YouTube from a Dartmouth presentation in 2011.
This is MUST viewing for vision neuroscientists and theoreticians.
high quality video editing, also.</p>
<p>
he shows famous, ludicrous 1966 letter from Seymour Papert (the Summer Vision Project letter: big laugh!)</p>
<p>
What is object recog... car, person, building, tree, lamp post: bounding boxes...</p>
<p>
time scale: abt 200 msec...</p>
<p>
David Sheinberg lab each fixation is bout 200 msec
and about 10 decr of visual field (from fovea)</p>
<p>
series of snapshots... ventral stream... core object recog...</p>
<p>
fast, feels effortless;</p>
<p>
large # of objects simult considered</p>
<p>
tolerant to variation</p>
<p>
current machines fail...</p>
<p>
hwo does the brain do it to underlie core object recog:</p>
<p>
want to build it in hardware...</p>
<p>
his lab: mainly systems neurosci;</p>
<p>
computer vision: test hypotehses;</p>
<p>
Rhesus monkey brain ventral visual stream;</p>
<p>
decision and action...</p>
<p>
neuronal spiking activity</p>
<p>
projected in about 100 msec to IT from pixel elvel that prjects to RGC neurons...</p>
<p>
what are the algos...</p>
<p>
what is the IT repres and can it support the object recog task...</p>
<p>
and how is the IT repr construcged?</p>
<p>
import of unsupervised learning...</p>
<p>
why is problem hard.... what is the hard part of the climb... the CRUX</p>
<p>
CAR... infinite # of variations... pose, illumination, clutter, interclass variation....</p>
<p>
population coding...</p>
<p>
one million RGS... one M D high D space...</p>
<p>
say pose changes thenit sweeps out another manifold...</p>
<p>
want a good separating plane...</p>
<p>
2 sheets of paper hopelessly tangled;</p>
<p>
how good is this repr in IT for doing visual recog.</p>
<p>
IT neurons many folks record in IT... listening to a neuron....</p>
<p>
he plays a neuron spiking...</p>
<p>
counting spikes within 100 msec with latency of 100 msec</p>
<p>
78 testobjects in 8 categories...</p>
<p>
0.5x size 2 degrees off center and 2X size...</p>
<p>
350 IT recording sites...</p>
<p>
how good is it for doing categorizaiton...</p>
<p>
spikes counts in 100 msec window...</p>
<p>
is it a face or not a face</p>
<p>
cat/dog vs toys vs food vs monkey face vs white box contours, hand/body; vehicles</p>
<p>
only small # of randly selected IT</p>
<p>
100 msec latency only...
passive viewing (no attn cues)</p>
<p>
is it better than V4 (Nicole Rust did it)... that was pic in 2009;</p>
<p>
it this neuronal code sufficient to explain object recog?</p>
<p>
take car and rotate and put onto difft bkgrounds...</p>
<p>
animals, boats, cars, cahris, faces, fruits, planes, tables</p>
<p>
vary it... car ID control variation</p>
<p>
get 29 benchbamrk... get fdifft d' = d prime performance.</p>
<p>
image psan abt 8 degrees....</p>
<p>
100 ms viewing, no past-mask, image spanned 8 deg at center of gaze...</p>
<p>
39 human sbujets
23K total image tests...</p>
<p>
DARPA collab: 96 channel arrays... triple 96 channel arrays.
lots of data over many days...</p>
<p>
5 images presented per second</p>
<p>
10 - 50 repetitions of each image, fuly randomzied...</p>
<p>
count spikes</p>
<p>
mean spike rate is about 70 to 170 spikes in a seoncd (count over 100 msec only)</p>
<p>
simult collect form 128 sites...</p>
<p>
how good is V4 vs IT...</p>
<p>
6 is a pretty high d prime
how good is this code...</p>
<p>
he shows perfomranc eof human subjects</p>
<p>
n= 29 points...</p>
<p>
x = perform of human subject
Y = putative code performance...</p>
<p>
terrible code</p>
<p>
DARPA happy code... does great even when huamns suck</p>
<p>
Neurosci happy code rough correl but not great...</p>
<p>
DARPA hapyy when humans are good...</p>
<p>
what codes are we testing...</p>
<p>
code basis: SVM based, or N elemenst, best element, Q90 element, median element</p>
<p>
pixel code: suck on invariance...</p>
<p>
pixel code is great for subtle changes in faces...</p>
<p>
code IT code with SVM... this is really good;</p>
<p>
58 neurons... good mean turing raiton 0.64 and consistency of 0.8</p>
<p>
things get better along ventral stream... IT based codes are the best.</p>
<p>
our data falsify all of these codes (ie human level is difft than these)</p>
<p>
IT weighted sum with an SVM is a human performance!</p>
<p>
how abt uncertainty due to neuronal sampling...</p>
<p>
just incr # of code elements to incr performance (but then consistency suffers)</p>
<p>
we estimate abt 500 neurons to get human performance...</p>
<p>
10M projecting neurons out of central and IT (potential # of population)</p>
<p>
linear readout codes...
the first spikes out of IT... passively viewing monkeys...
trying to minize feedback ...</p>
<p>
just measure shape variables...</p>
<p>
you still have info on position, size...</p>
<p>
Invariance is the CRUX problem</p>
<p>
untangling object identity manifolds...</p>
<p>
IT are nearly sufficient to explain human perf but NOT V4...</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>