SAENKO. Kate, Virtual Object Detectors, U Mass, Aug 6, 2014
<p class="MsoNormal" >SAENKO. Kate, Virtual Object Detectors, U Mass, Aug 6, 2014
want to go from 3d to 2d directly;
go from a virtual backpack (illustration) eg to a 2d model...
motivation: where is the mug in this picture, eg detector fails
eg by using DPM model (parts model)
problem is that the particular pose of the mug in that partic picture
is not anywhere in the training data;
poses are usu. human centric....
<b>so, rather than getting training data from the web,
we want to get synthetic data from crowdsourced CAD models
so, you can make whatever pose you want using the model</b>....
is this picture real or fake? ( a little game we play)
shows a dirty police car (looks real but is not!)
cars: can get some great models for that....
bad news: photorealism is hard to achieve
must model albedo, bdrf, etc.
most cheap models do not do shadows, lighting etc.
good new is that we don't need photorealism;
model what we have and ignore the rest
model 3d shape, pose (and ignore scene, textures, color)
so, how do we do it....
we do have 3d shape and pose;
do not have light direction,
so, how do we ignore the diffs btwn the non-realistic images and the real test images...
Answer: domain adaptation...
start with discrim scoring function f (w)...
keep regions of image with high score
using Hoff features.
inspired by Hariharan, Malika and Ramanan in ECCV 2012...
this is textbook LDA = linear discrim analysis...
assume generative model... using prob of model given feature...
Main idea: ...
shows mean HOG vector of bicycle... does better by improving covariance matrix.
assume covariance is the same for all classes...
(for the real images)
go to trimble 3d warehouse...
these are models created by random people of 3d objects...
office dataset has 31 objects in it (eg lamp, bike helmet, bike, coffee cup, etc)
take gen images and place against a white bkgrnd...
just get shape of object (as gray object)
test on webcam data... eg of a keyboard vs
dSLR camera vs amazon.com website images of objects...
and use imagenet... 150 to 2000 images...
31 categories: file cabinet, head phones, etc.
virtual gray gives better stats than virtual images alone.
dSLR does really well (cuz its so similar to the target domain (the webcam))
virtual gray was 2 models with 15 poses each...
compare to Goehring, Hoffman et al in ICRA 2014
Interactive adaptatiion of real-time object detectors.
we were able to match training with images from Imagenet
(using our synthetic virtual models)
Related Work: skip that...
Future Work:
collab with Berkeley LSDA large scale detection thru adaptation...
ImageNet is now the largest source of labeled training data
wach image (of 20k) has a single label
and a subset have bounding boxes also.
so, we want to make unneces the use of bounding boxes...
detectors also localize objects in an image...
want to learn the transformation from classifier to a detector....
uses deep conv neural network...
from these we get out 10K detector...
in detector network question is... is a bounding box simply
background or does it contain one of the classes.
Oracle means the ideal situation in which we have detectors for all 200 classes.
we go from 10 to 15.9 by doing adaptation....
she shows images that were classified incorrectly... they still look like intuitive errors.
have since trained on 7.4K images from YouTube.
</p>