TSAO, Doris, Face Recognition, Caltech, June 4, 2018

<p class="MsoNormal" >TSAO, Doris, Face Recognition, Caltech, June 4, 2018

This is at 5:10PM at Stanford;&nbsp; the MBC&nbsp; trainee seminar.

Doris is a prof. of neurosci at Caltech.

she begins:

first slide:&nbsp; the Chauvet cave paintings:&nbsp;&nbsp; bears

what is the neural code for visual objects (eg bears)?

she focuses on the MACAQUE brain.

same areas as human vision:&nbsp; V1 to V4 and IT = inferotemporal cortex;

thence to perirhinal cortex.

there is a code for faces in IT cortex&nbsp; (Chang and Tsao, Cell 2017.)

Charlie Gross:&nbsp; 1980s&nbsp; discoverer of face cells;

but elicited much skepticism,

then Nancy Kanwisher, J Neurosci in 1997.

faces versus&nbsp; objects:

there are <b>six face&nbsp; patches</b>:

for example ML and

anterior most = AM..

<b>

MF = middle face patch</b>... tuned to head orientation.

face patches and faces...

<b>want to reconstruct the human faces that the monkey sees

(based on responses in the six patches)....

</b>

eg&nbsp; analogy to colors...

eg&nbsp; 3 axes for color...

what are the axes for faces.

Cootes in 2001.

we&nbsp; did PCA on many faces

n columns by m rows.

take shape PCs (principle components) and 25 appearance PCs.

first PC for appearance.

50d space.

appearance is shading&nbsp; (gray)

shape... oval versus circle etc.

ramp-shaped tuning.

weighted avg of features.

Rn =&nbsp; Sij * Fi

50 face features.

<b>she reconstructs the faces from the responses of&nbsp; 205 face cells... wow</b>!&nbsp; v close.

linear projection.

projects a face onto 50&nbsp; featues.

generation of metamers.

2000 faces;

summary:

simple operation: project face onto specific axes in face space.

*

IT&nbsp; cells&nbsp; performing linear projection

Yamins and DiCarlo PNAS 2014.

shows face mapped to an emoji&nbsp; ( a metamer)

Q1&nbsp; how are monkey faces coded?

Kati Gothard studies this...&nbsp; (monkey faces)

Q2:&nbsp; profile face encoding?

cells in AM are view invariant...

profile exactly correlated with frontal view.

Q3.... our luck was in using&nbsp; shape and appearance.

(rather than pixel intensity)

Q4 can you only reconstruct faces in 50d face space?

next expt:

faces off the internet...

we get shape from&nbsp; points around periphery.

encoding accuracy..

using 32 params only.

158 cells...

AlexNet is quite good&nbsp; (just learning edges etc)

interim summary:&nbsp; we've demonstrated a <b>feedforward code for face identity</b>.

so,&nbsp; FACE patches... got it... but

how about the rest of IT cortex.

and are Faces a neural Rosetta Stone?&nbsp; (or&nbsp; are they unique?)

how about bodies, scenes, and colored objects?

how about other&nbsp; specialized networks?

we went into noman's land.

electric microstim + fMRI...

stim set expt:

vehicles etc.

houses, fruits,&nbsp; animals, houses...&nbsp; airplanes,

MLO cell and AMO cell.

O above = object.

a new object network in IT network across animals.

body patch network...

no man's land is distinct from body network.

is it just oblong, thin objects?

ALO and AMO... high aspect ratio.

hypothesis:&nbsp; objects are coded along axes.

fact 1&nbsp; (per Yamins)&nbsp; 4096&nbsp; coding.

axis is the top PC.

penultimate.

50 object features...

eg&nbsp; take image of an airplane and project it onto 50 object features.

from 400 IT cells&nbsp; (from no man's land and from body patch network)

IT just cares about form...&nbsp; but not category.

only from <b>400 cells in monkey</b>....

IT cortex...&nbsp; some specialized for faces but...

yes... distributed:&nbsp;&nbsp; can use just 400 cells...

but these 400 cells came from no mans land.

resolution:

any given object has nonzero projeciton onto specialized areas.

last part

so far,&nbsp; only&nbsp; feedforward.

with six patches.

hierarchical Bayes inference.

brain has a generative model.

prob of (pixels given David Hubel)

where does top down generation and feedfwd come together...

answer is <b>Perirhinal cortex</b>....

code for faces in perirhinal cortex.

retrograde tracers:

go to perirhinal; claustrum and ...

perirhinal is crucial.

perirhinal... has a high conc. of face cells.

2000 faces...

top four...&nbsp;&nbsp; familiar&nbsp; faces...

and not just ALL familar faces... just four.

ramp-shaped tuning with one particular&nbsp; lab manager (that elicited recognition in the monkey).

perirhinal face patch is where top down and bottom up are combined... she thinks.

<b>perirhinal..&nbsp; may have both semantic and shape and appearance axes</b>.

prob&nbsp; (shape, appearance&nbsp; and given category)

interim:

perirhinal face patch:

Marr&nbsp; GM grandmother cell

yes...&nbsp; cuz embedded&nbsp; in hierarchical&nbsp; face network.

perirhinal...&nbsp; she thinks it receives the info from&nbsp; IT.

retrograde tracer injected into&nbsp; IT goes to perirhinal.

(ie&nbsp; perirhinal is projecting to the IT)

&nbsp;

&nbsp;</p>