ALVAREZ, George, Multi Object Perception, Harvard, Jan 15 2015
<p class="MsoNormal" >ALVAREZ, George, Multi Object Perception, Harvard, Jan 15 2015</p>
<p>
this is at 4PM in Margaret Jacks Hall (adjacent to my old office) at Stanford.
He is an Assoc Prof at Harvard.
full title: <b>How representing multiple objects (and features</b>) as an <b>ensemble</b>
<b>enhances higher level visual cognition</b>.
ABSTRACT:
The human visual system has a massively parallel architecture, and
yet it can only accurately represent a handful of objects at once.
A vast literature has focused on ways to explain these
<b>severe capacity limitations</b> (slots, resources, central bottlenecks, brain juice, neural competition), but today I will focus on understanding<b> how the visual system copes with these limitations</b>.
Specifically, I will focus on the possibility that the visual system does what we (scientists) would do with<b> a bunch of noisy measurements: it averages them </b>and combines them to <b>represent a statistical summary, or ensemble representation</b>.
I will describe recent experiments which show that <b>ensemble representations</b>
can be extracted at early processing stages (prior to object or scene recognition)
and that these <b>statistical representations enhance downstream processes</b>,
including <b>attentional guidance, object recognition, and perhaps categorization</b> as well.
Combined, these results suggest that <b>ensemble representations</b> are efficiently computed and useful for many stages of visual processing, making <b>ensemble coding a crucial mechanism</b> for coping with the severe capacity limits on higher-level visual cognition.
he begins....
ok: our visual perception is v rich and yet it is incomplete!
He starts off with great demos of <b>change blindness</b>.</p>
<p>
many many explanations of this from from difft groups... he lists them</p>
<p>
three benefits of ensemble representations... and now... how does it work?</p>
<p>
1) does it use summary statistics...</p>
<p>
see Dan Ariele...</p>
<p>
see Cahong and Treisman in 2003 on Mean size and mean location eg Alvarez and Oliva 2008</p>
<p>
mean expression in Haberman and whitney in 2007</p>
<p>
so, that viewpoint advocates that the scene collapses down to a single measurement
(NO! that's wrong... that's too sparse)</p>
<p>
how about spatial layout statistics?</p>
<p>
they looked at <b>22,000 natural scenes</b>...</p>
<p>
<b>the global receptive field accounts for the most variance</b>...</p>
<p>
(a global feature might be one that judges <b>what category the scene is in eg)</b></p>
<p>
do you have a statistical <b>GIST representation of the scene</b>?</p>
<p>
his task; focus on fixation cross and then<b> track 4 dots that are circulating around like flies randomly </b>foraging amidst a group of other moving dots...</p>
<p>
(ie you have multiple foci of attentyion)</p>
<p>
for this talk...</p>
<p>
he will refer to the "hot spots: that are the focus of attention...</p>
<p>
if some of your attn is attending to one of these dots then
are you better at seeing changes in the vicinty of that dot ,ie
(would your performance in a change blindness task improve locally adjacent to the dot)
E1 layout statistics without attention...</p>
<p>
you do a change detection task when image is rotated by 45 degrees. in a local neighborhood of the focus of attn (of one of the dots)</p>
<p>
doing attentional tracking task with gabor patches in the bkgrnd...</p>
<p>
for 17 ms and then 200 ms gap...
then report... change or no change...
did global pattern of gabors flip or was there just a flip locally in one gabor...</p>
<p>
he shows d prime on y axis...</p>
<p>
main task is attending to the moving dots...</p>
<p>
we see much better perception when the overall ensemble changes
(all the gabors flip by 45 degrees.</p>
<p>
Q abt crowding... A yes could be a factor...(but just within a small region)</p>
<p>
they have poor detection when layout is the same...</p>
<p>
so, it appears from this the <b>princ component number 1 is the global receptive field</b>.</p>
<p>
how about L vs R.. (a weaker feature)</p>
<p>
we do see some sensitivity to L versus R)</p>
<p>
but the <b>major global feature is the top/ bottom structure</b>.</p>
<p>
so, we have sensitivity to patterns in the background...</p>
<p>
now...</p>
<p>
can that spatial ensemble be used to tell you where to attend next?...</p>
<p>
task : is there a person present in the image...?</p>
<p>
but... where do you look in this image? (there is a probability map or a heat map
that will show you the likely spots for a person (like where is waldo?)</p>
<p>
a Subject might say "I recognized the scene and then I looked where there might be people..."</p>
<p>
vs I dunno.. maybe its just ensemble statistics...</p>
<p>
then we took these images (we took real world scenes and then transformed them
into unrecognizable textures.</p>
<p>
so, how can we use this to test use of statistics.</p>
<p>
a pedestrian was shown in a high-probability location...</p>
<p>
find him and ask.. is he facing L or R...</p>
<p>
eg put his image in upper L corner or in a low probab location...</p>
<p>
we tried to address the visiblity of the pedestr image by incr intensity/ contrast ...</p>
<p>
we find <b>big advantage to find pedestr at the high probabiity locations</b> based on
REACTION TIMES... (takes only 2 seconds at high prob locations vs 2.6 secs at low probab locations)</p>
<p>
Jay Q: are these genl statistics versus scene specirfic for a particular scene
A can I say.. a little bit of both...</p>
<p>
ruling out poss low level compounds...</p>
<p>
1) absol screen position...
eg is it simply best to place the pedestr starting in upper L...
(we adjusted that by counterblancing locations across images...</p>
<p>
Jay Q .. is it just relative size, eg (say compared to a pillar on a bldg.
A... maybe...</p>
<p>
2) confounder # 2... how abt just comparing low prob locations with no bkgrnd...
NO diff when we test that confounder...</p>
<p>
So, is this really abt natual image statistics?</p>
<p>
natural layout stats are highly dependent on orientation !!!
(can we eliminate the advantage of high prob locations by rotating some bkgrnd features)</p>
<p>
conclusion... <b>benefit for high prob location is much greater in natural upright alignment</b>
(ie if local bkrnd is rotated it does diminsih the effect but it is still a high prob location)</p>
<p>
comment: this is a real evolutionary advantage... having your attn drawn to high prob locoations
(but... say, is it difft for squirrel locations versus people locations...
is it specific to surfaces etc. or is it based on where squirrels usually are, eg)
A we'd like to do that...</p>
<p>
now... scene guidance without scene recognition...
as in Torralba in 2006 (inspiration for some of this work)</p>
<p>
scene context guides attn in real-world scenes...</p>
<p>
(even in images that cannot be recognized.. as in above expts)</p>
<p>
support for 2 pathway models of visual search...</p>
<p>
"global pathway is an automatic implicit link btwn early stat properties and
learned regularities as in Chang.</p>
<p>
Summary of part one of the talk:</p>
<p>
spatial layout stats are effic represented...and do help to guide attention...</p>
<p>
part 2*</p>
<p>
3 benefits of ensemble repres.</p>
<p>
might you use ensemble reps to enhance recognition of individ objects? YES.</p>
<p>
scene context might be useful for object representation...
shows a bathroom sink in a fuzzy pic..</p>
<p>
and a hand drill on a work bench in a v fuzzy pic.</p>
<p>
is this "cognitive/ strategic) or is this
perceptual and automatic (lower level) pattern match versus
reasoning like oh that's a bathroom sink so that adjacent blob might be a hair dryer.</p>
<p>
scene context and object recog.</p>
<p>
new task... eg recognize a football palyer superimposed inside a church</p>
<p>
or pope on the football field (harder to recog these out of context)</p>
<p>
are the people with greater sensit to ensemble change also
better facilitated in object recog by consistent context...</p>
<p>
(pope in church and FB player on the FB field)</p>
<p>
our result: we get replication of the main effect...</p>
<p>
ie ensemble change benefit is correl with
scene-context benefit (seeing pope in church).</p>
<p>
(in ensemble change benefit... he uses d-prime as measure and uses
of errors in the scene context consistency task)</p>
<p>
we got strong correl
btwn these 2 tasks...
r = .43....</p>
<p>
so part 2 summary...</p>
<p>
YES... these are correlated (the 2 tasks)</p>
<p>
how abt just using texturized bkgrnds instad of detailed and accurate bkgrnds...</p>
<p>
the spatial struc does matter but there is still an effect even when bkgrnd is texturized...</p>
<p>
so, this points to some perceptual/ automatic component to this
(rather than simply using cognitive reasoing
(a la popes are usu seen in church and not on FB fields)</p>
<p>
texturized means (instead of displaying the crowd at the stadium of a football field..
you just display blurred blobs of heads at the right vertical postion.)</p>
<p>
next... bootstrap of vision...</p>
<p>
intro Hubel and Wiesel style hierarchy of vision...</p>
<p>
later on see Biederman Geons as in Bied 1987 or</p>
<p>
he shows two images from freeman and simoncelli
one with detailed bkgrnd and the other with smoosh... fuzzed out black/ white blobs.</p>
<p>
replace things like couches by blobs that preserve some low level aspects of image.</p>
<p>
task: compare modern office chair to FERRET eg</p>
<p>
this is a stroop like task...</p>
<p>
result ... if congruent then faster to recog animal + ferret versus chair + ferret (which is slower to recog)
and effect is preserved if you use a blob instead of the chair...</p>
<p>
bundle of features like this is suffic to prime the system...</p>
<p>
so, 3 benfits of ensemble repres</p>
<p>
1) attn guidance</p>
<p>
2) facilitates object recog and</p>
<p>
3) guides categorizaiton of objects...</p>
<p>
so, this is precision without detail !!!</p>
<p>
from visionlab at harvard...</p>
<p>
so, how do ensembles enhance visual cognition</p>
<p>
<b>they guild deployment of focal attention</b>
<b>and facil object recognition</b>
<b>and they enhance indiv item reps in working memory..</b>
<b>and they are early perceptual correlates of broad categories</b>...</p>
<p>
Jay.. perhaps people can recogn objects thruout the visual scene
(even with things in the periphery)
Jay: you're priming effect tells me that some recog is occuring...</p>
<p>
A Jim Di Carlo can do tricks by flipping the object btwn being noticed in the periphery and foveation. (can see parts v clearly when foveated)
A I think you need detailed features to learn these associations...</p>
<p>
aud Q it'd be great to probe S's causal model of the scene
ie... was this blurred because of wind coming from a certain direcition, etc.
try to manipulate construals of S.</p>
<p>
so, what have we learned abt vision outside of focus of attn.
is it just that the info is degraded outside focus of attn..</p>
<p>
A I was not trying to model the process but merely saying that the bkgrd is helpful...
A dipping a toe in the water on this but too early to say...</p>
<p>
Jay... this is a characgteristic of scenes
(vertical layout with ground or floor at bottom, then the focus of attn on the object and then
sky/ ceiling above, eg.</p>
<p>
</p>