Malik, Jitendra
<p>Sep 14, 2016 09:04 PM</p>
<p>
MALIK, Jitendra, Computer Vision, CAL, April 15, 2016</p>
<p>
intro by Silvio Savarese (of CSD here)</p>
<p>
this is a special seminar in CS; at Clark Aud at 11AM at Stfd;</p>
<p>
he says "object detection is nearly solved!</p>
<p>
(he was a PhD student here years ago);</p>
<p>
opening slide shows crowded room of kids at birthday party ...
from MSFT Research: with a dozen bounding boxes around kids; knife, wine glass; book; chair;</p>
<p>
refs from ICCV2015 with papers by Kaiming and He from arXiv 2015
and Shaoqing Ren from NIPS 2015.</p>
<p>
start with Hubel and Wiesel in 1962 discovered orientation cells in cat cortex.</p>
<p>
then 1980...
layers of computation</p>
<p>
in Biol Cybernetics 36, 193 in 1980
paper by Kunihko Fukushima</p>
<p>
Neocognitron: A Self-Organizing Neural network Model etc.
(from Tokyo)</p>
<p>
intended app was hand-writing recog.</p>
<p>
then Conv Neural Nets by LeCun et al
used backprog to train weights in the architecture.</p>
<p>
biggest impact was from Geoff Hinton et al in mid 80s...</p>
<p>
handwriting recog...</p>
<p>
and applied in sliding window paradigm for face detection in 90s.</p>
<p>
he got a phone call from GEoff Hinton... why don't you like deep learning? Geoff asked Jitendra...
what would convince you? Geoff asked. (Jitendra... unimpressed by MNIST... too easy... everybody can get 100%)</p>
<p>
R-CNN /Girshick Donahue Darrell and Malik in CVPR in 2014...</p>
<p>
we had ImageNet, Mech Turk (all rest on a tower of pygmies (if not the shoulders of giants))</p>
<p>
Multibox from Google</p>
<p>
Girshick Fast R-CNN...</p>
<p>
HOG DPM is prehistoric (like Neatherthal Man! he jokes)</p>
<p>
latest is REsNet (even faster RCNN) 86 layers</p>
<p>
just keep going deeper and deeper...l</p>
<p>
HOG was shallow; AlexNet had 8 layers and VGG had 16 layers</p>
<p>
and now ResNet has 86 layers...</p>
<p>
he shows scores on PASCAL VOC 2007 Obect Deteciton and REsNet has 86% score!!!!</p>
<p>
so,, are we done....no!</p>
<p>
he shows woman playing accordion (street musician) with hat gathering money...</p>
<p>
Asian man contemplates giving her money...
she sits on a bench...</p>
<p>
another woman is walking away...</p>
<p>
will person B (the asian) put money into the musician's tip hat?</p>
<p>
3d reconstruciton from a single image</p>
<p>
by Kar et al and Malik in 2015...</p>
<p>
take single image of a car and then do segmentation (this pixel is in car, and this is not)</p>
<p>
and then viewpoints and pose and keypoiints
as in Tulsiani and Malik in CVPR 2015...</p>
<p>
perhaps it helps if you have man other photos of cars (not nec. the one in the photo)</p>
<p>
get say non-rigid structure from motion... deformable 3d model...</p>
<p>
get basic 3d models eg of</p>
<p>
airplane, bicycle, spaceship, motocycle...</p>
<p>
so, pure feed-forward view of processing ventral stream with layers of simple and complex cells led to
the neo-cognitron and then to CNNs.</p>
<p>
but ventral stream is far more complex... it is bidrectional with feedback connections..</p>
<p>
he says "I am interested in feedback models...</p>
<p>
shows Felleman vanEssen diagram...</p>
<p>
</p>
<p>
say you want to find joints of human as with Xbox...</p>
<p>
posture estimation in humans....</p>
<p>
start with standing stick figure... what correction must be applied to correct it
and do that iteratively til correct.</p>
<p>
he calls this Iterative Error Feedback IEF for 2d human pose estimation.</p>
<p>
he calls these structured output problems (not simply object classification)</p>
<p>
need to get relationships btwn head, shoulder, elbow, wrist, hip...</p>
<p>
must do this quickly and approximately...</p>
<p>
(must be fast; if it take 5 mins to figure out... that's a tiger... you're dead! he jokes)</p>
<p>
shows woman doing situps on a bean bag chair... program gets it, and matches her pose (including legs, hips, etc)</p>
<p>
we need good methods for tracking people...</p>
<p>
but can we apply to other apps..</p>
<p>
hypersegmentation... color just the pixels in eg the image of a partic. person in a photo.</p>
<p>
calls this iterative instance segmentaiton...</p>
<p>
starting from hypercolumn architecture as in Hariharan.</p>
<p>
iterative approx to refine heatmap predicitons...</p>
<p>
use prev. iteration to improve prediction..</p>
<p>
</p>
<p>
where are things and WHAT are they? must solve both...</p>
<p>
situate in 3 space...</p>
<p>
must think in 3d...</p>
<p>
amodal perception (from psych... the parts that are hidden... filling those in)</p>
<p>
he has a paper written by Euclid (English translation)</p>
<p>
amodal instance segmentaiton...</p>
<p>
he shows donkey... you also want the amodal segmentation also
(ie want to see "in your minds eye"
the parts of the donkey that are occluded by the rider.</p>
<p>
use amodal eg of bike to predict its appearance when rider is in front of it, occluding it.</p>
<p>
*8</p>
<p>
Contextual action recog...</p>
<p>
to do it... we need recog of parts etc.</p>
<p>
R*CNN on PASCAL VOC Action (database)</p>
<p>
red bnd box is always abt a person...</p>
<p>
green box highlights area of interest...
eg making a cellphone cal)
or playing a cello..</p>
<p>
R* means region (and here we use multiple regions)</p>
<p>
so, back to Asian person contemplating giving a tip to street musician.</p>
<p>
shows book: The Nature of Explanation</p>
<p>
Kenneth CRAIK...</p>
<p>
he shows a great quote...</p>
<p>
if organism carries a "small-scale model of external reality and of its own
possible actions within its head, it is able to tryout variour alternatives, concluee which is best,
react to future situations before they arise,
utilize the knowledge of past events in dealing with present and future
and in every way to react in a fuller, safer and mroe competent manner to the
emergencis which face it.</p>
<p>
from CRAIK in 1943 Ch 5 page 61.</p>
<p>
he says... modern control theory (a la Kalman) uses a state space formalism to achieve this.</p>
<p>
so, how to apply this...</p>
<p>
eg with robots...</p>
<p>
how will enviro look after I interact with it...</p>
<p>
eg playing billiards..</p>
<p>
</p>