SHAKHNAROVICH, Greg, Cascade Visual Image Segmentation, TTI Chicago, Nov 4 2013
<p class="MsoNormal" >SHAKHNAROVICH, Greg, Cascade Visual Image Segmentation, TTI Chicago, Nov 4 2013</p>
<p>
Cascade philosophy</p>
<p>
predict Y from x...</p>
<p>
often can be formualted as selcting one or a few from a v large set of possible solutions
but hard when soln space is large and complex</p>
<p>
pruning solution space might make decision easier</p>
<p>
cascade: keep pruing solution space</p>
<p>
/Semantic Image segmentaiton</p>
<p>
3 problem defintions</p>
<p>
genl def of segmentaiton: parition an image into meaningful regions</p>
<p>
semantic segmentation: label every pixel with a category and or instance</p>
<p>
Figure ground segmentaiton: only 2 categories</p>
<p>
we will discuss all three;...</p>
<p>
see CVPR 2013 with Zhile Ren</p>
<p>
and CVPR 2013 with Payman Y... et al..</p>
<p>
</p>
<p>
non-sematnitic segmentation</p>
<p>
joint work with Zhile Ren cvpr 13...</p>
<p>
task: partition image into measningful regions</p>
<p>
our task: partition the image but no category labels</p>
<p>
but multiple ground truths are possible...</p>
<p>
runners (how abt bndry btwn shoe and leg or shorts and leg...
multible observers do difft segmentations.</p>
<p>
so task is ill-posed but useful eg for labeling</p>
<p>
2 regimes:</p>
<p>
superpixel regime : produce hundreds of small regions;
main gola is to preserve true boundaries but you are over-segmenting...</p>
<p>
Versus: alrge segment regime: few (say, < 50) high-quality segments...</p>
<p>
ideally its hierarchical segmentaiton</p>
<p>
challenge is the trade-0ff btwn number of segments and their quality...</p>
<p>
state of the art for superpixels is SLIC algo (constrained to contiguous simialr colors eg then join)</p>
<p>
versus ERS algo (similar problems: when you go to larger segments they can cut across objects)</p>
<p>
state of the art for large segment regime: gPb-owt-ucm...
boundary map can be thresholded at different levels
(this is boundary detection)</p>
<p>
our approach:ideas</p>
<p>
scale in prior methods is controlled by either # of superpixels or tweaking some threshold</p>
<p>
what works for one part of image often breaks down for another part</p>
<p>
our gola: start with fine scale and then merge regions but do not over-merge..</p>
<p>
agglomerate clustering of regions...</p>
<p>
preserv e boundary recall...</p>
<p>
the basic object we reason about... is a PAIR of regions with a common boundary (neighbors)...
decide whether to merge or not...</p>
<p>
problem setup...</p>
<p>
take image I and initially segment it into regions R
and neighbors N.
then compute features on them and decide whether to merge or no...</p>
<p>
say 10 people look neighbors... how many think they sould be grouped vs not...</p>
<p>
but there are mutiple ground truths...
train this on thousands of examples</p>
<p>
use logistic regression</p>
<p>
weigh 2 kinds of mistakes (group together things that should not be grouped and vice versa)
using log probabilties...</p>
<p>
simple greedy algo...</p>
<p>
algo: strat merging from the highest probs, then update featrues and Pg as we merge</p>
<p>
But... a problem... if u train on tiny regions... then stats do not apply to large regions...
(cannot use same classifiers/ features...)
so this is wehre cascade comes in...</p>
<p>
CASCADE</p>
<p>
merge for awhile</p>
<p>
then stop and re-train a new merging model and then go on merging...</p>
<p>
genl principle: train stage t on ouput of stage t-1.</p>
<p>
alpha = prob of making an erro</p>
<p>
recall is monotonic with alpha...</p>
<p>
(he cannot recover from errors... therefore want an alpha that minimizes errors</p>
<p>
but want to make sure you do not throw out the true positives (that u really want to merge)</p>
<p>
features: color histogram, texture, gradients
shape features: convexity, size, location
boundary features: globalized evidenc eof boundary gPb</p>
<p>
wind up having abt 60 to 70 features in the vector...</p>
<p>
geometrix context (Hoiem et al) 32 bin histogram for each label (eg sky, vertical etc)</p>
<p>
color histograms...</p>
<p>
Shape and boundary features...</p>
<p>
use SIFT when regions are large enough to captures many kepoints...</p>
<p>
shape features: differences in area, perimeter, convexity, Euler number
convexity of merged region vs the orig two....
</p>
<p>
Experiment Results</p>
<p>
ISCRA image segmentaiton by cascaded region agglomeration</p>
<p>
we train 60 stges of ISCRA on data set...</p>
<p>
we win or tie compare to others..
results on ASA</p>
<p>
BSDS300 ISCRA is top... BSDS500 wins (say, compared to SLIC, ERS, Hoiem, gPb-owt-ucm)</p>
<p>
he alsho shows results on MSRC, etc.</p>
<p>
our innov: adaptive retraining of weights (say compared to WS1)</p>
<p>
results on OSI</p>
<p>
Part 1 concluciosn</p>
<p>
can get high achievable accuracy with signif fewer regions</p>
<p>
8
Semantic Segmentation state of the art...</p>
<p>
label every pixel with a category...</p>
<p>
eg Pascal VOC 2012 got 46.5% was the state of art; we improve this to 48.1%.</p>
<p>
eg animals, furniture
prior state of art was approach by Carreira et al.</p>
<p>
seq of steps:
produce a pool of foreground segments and
train predictors forr score (per segment and class) then
greedily assmble labeld segmentaiton by "pasting" foreground segments
with highest scores.</p>
<p>
our approach: theideas...is above...</p>
<p>
random field setup...
defer classifiction...</p>
<p>
CRF = conditional random fields or structured hinge loss (struc SVM)</p>
<p>
MAP = minimla energy = highest score...</p>
<p>
coputer x star = argmin (over x) for a weighted sum over energies.</p>
<p>
diverse M-best algo</p>
<p>
(looking for solutions that have high energy but are quite different (diverse) from the best solution)</p>
<p>
see ECCV 2012 Batra et al Shakhnarovich...</p>
<p>
in our expts we compute diversity using delta as the Hamming distance btwn assignments to superpixels or pixels...</p>
<p>
using this we got a big gain in perf on PASCALaccuracy...</p>
<p>
he calls Oracle the algo that somehow knows the best M solutions in hidsight...</p>
<p>
ranking vs inference</p>
<p>
results (for part 2: semantic segmentaiton: expts)</p>
<p>
VOC 2012 test accuracy
</p>