Computer Vision Sebastian Thrun AI Course
<p>Dec 25, 2011 04:31 PM
 
this is from the Norvig/ Thrun course...
 
CV means making sense out of camera or video data;
He shows his heavy Nikon D3 camera;
he mainly uses his Motorola cell phone camera...  has cameras front and back...
 
all cameras have a lens and a chip...
 
Image Formation...
start with a pinhole camera...  person projected thru the pinhole onto the image plane (the projection plane
or the camera chip);
 
X = big X = height of person and
x = little x = height of image
Z = distance to the person
f =  focal length from lens to chip;
 
similar triangles
 
X/Z=  x/f;
 
or   x = W *  f/Z;  central law of perpective projection;
 
note: that cannot tell whether an object is small and close or far and big !
he shows this with a toy helicopter  (a toy of his young son);
 
same is true in the horizontal direction
 
Vanishing points....   railroad tracks going to infinity to vanishing point
 
street corner with buildings (house facades)...  get  TWO vanishing points....
 
(can have an infinite number)  eg  a many sided cube like an octagon....
 
LENSES....  trouble with pinhole....  light is not bright enough...  (also get diffraction as it goes thru the pinhole)
use a lens to gather more light  dbl convex...
 
but note:  it is only in focus at one object plane....  must change focal length to get it into focus;
 
Computer Vision...
classify objects
3d object reconstruction
motion analysis
 
Invariances:  to scale, rotation, translation, illumination, deformation, occlusion, and view point (where r u looking from)
top of the helicopter or bottom of it...
 
invariances are crucial to CV...
 
Greyscale Images....  he uses an image at a draw bridge in Amsterdam... he has  cables going up to the top and
girders;
 
he shows matrix of grey scale intensities..
 
255 =  very bright
0 = black (no light)
 
he shows   4 by 4 matrix  with a light region at the top and dark at bottom
 
? how to find the edge ?
 
Use a <b>LINEAR FILTER</b> ...
 
easiest <b> feature detector </b>is a 1 by 2  matrix  (a horizon vector) =  a mask in which
we subtract  the column on the left from the column on the right.
apply the mask to a 4 by 5 matrix and you get a 4 by 4 matrix which is the result...  it shows the transitions...
this is our first  <b>KERNEL</b>...
 
GRADIENT IMAGES...  convolve the original image with the kernel (or mask)  get the gradient image
 
Ix (eye sub x) =  I X  kernel    that's a cross  (<b>convolution operator</b>)
 
can do  both horiz and vert edges silultaneously  ie  Ix and Iy
 
<b>CANNY edge detector</b> =  for JOhn Canny at Cal Berk... ...  nicely traces  edges by connecting them..
 
few other common masks in the literature
 
SOBEL mask;  Prewitt mask;  Kirsh mask...
 
<b>GAUSSIAN MASK</b>...   a hill or breast....  fall off is the same in all directions... rotationally symmetric....
it BLURS the image  (noise reduction... less sensitivity to bizarre indiv pixels)
 
<b>HARRIS CORNER DETECTOR </b>...  finds a lot of horizontal and vertical gradients...
if  both the veritcla and horizon edges fire... then it fires....
 
rotates image with eigenvalues   use eigen value decomposition...  ?  want to minimize the off diag elements ?
 
Modern Feature Detectros...  localizable,  invariant to lots of stuff
 
<b>HOG  histogram of Oriented Gradients</b>..
 
<b>SIFT =  Scale Invariant Feature Transform </b> sift  feature extractor..
 
look at HOG or SIFT...  everyone preprocesses images by feature extraction....
 
*
Dec 25, 2011 05:12 PM  This is the Second lecture by Sebastian on CV....
 
<b>This class is about 3D  vision</b>
 
<b>starts with pinhole camera...</b>
 
<b>try to recover the full 3d image from the 2d image  including the depth of the object;</b>
 
STEREO  quiz...  given 2 identical cameras...  can we now recover the depthof a scene...
 
say we have TWO pinhole cameras (R and L eyes) projecting onto two image planes...
 
say person P  is projected as a height x1 and x2  (two images)
 
note:  two triangles  ... one for each eye... bring them together and u get one single triangle...
can derive formula  which is  z =  fB  /  (x2 - x1)
 
B = baseline =  interoptic distance (my name for it)
Z = distance to the object
 
(P is a single point)...  derive depth Z (ie RANGE)  from  f = focal length
B = baeline and  diff btwn  x1 and x2 = (x1 - x2) =  delta x...
 
the CORRESPONDE Problem...  where is the point P that is seen inone image  in the Other image???
how do u line them up???
 
determination of correpondence.......  if u get it wrong ,   get phantom points...
 
ssd = sum of square difference error:   pick the place along a horiz line that has the smallest ssd
(using a small square patch around the point of interest... sum over all the pixels....
compare eg a template (matrix ) based on the L image, then sweep it along a horiz line in the R image to
get the correct alignment...
 
he shows disparity map from Stanley robot car... black means don't know corresp... white means yes.
 
does it make sense to look at context to determin  alignment.... yes , obviously....
 
try to match the whole scan  (trying to match L to R images)
 
use a COST function to minimize cost of alignment 
 
Dynamic Programming... hwo to compute the best alignment of R and L.
problem is order N2 ....
 
path that alignment program takes depends on the costs assoc with matching two pixels and costs of occlusion
and non-matching..
 
James Davis... Honda Research now at UCSC...   add texture to the scene... a striped pattern on
unequal distacnes...  easier to disambiguate the correspondence of points...
structure light really helps stereo vision.
 
MSFT Kinect...  uses laser to add texture to recover depth...
uses stereo vision....
 
many difft techniques..  time of flight...
 
they use laser range finders  LIDAR... gives v good  depth estimates...
 
*
*
PART III  lecture 3 by Sebas. T  on CV...
 
Dec 25, 2011 07:33 PM
 
<b>CV III  Motion...</b>
 
<b>Structure from Motion...</b>
 
<b>how to capture 3d world  from the locations of the CAMERA...  put it into different places and</b>
<b>then combine the images to get 3d data on the object...</b>
 
 
Carlo Tomasi and Takeo Kanade  1992....  could get every single corner point of a house
which he rotates... amazing... using a Harris corner detector...
 
could fly over Pennsyl and get 3d terrain maps...
 
Marc Pollefys ... Swiss...   reconstruct the 3d shape of the bldg facade...  v impressive for its time...
zooms into bldg model....
 
holy grail of 3d computer vision is total reconstruction...
 
SFM = struc from motion math...  he shows rotation matrices...
use nonlinear optimiz techniques
 
gradient descent; conjugate gradient; guass-newton  etc.
 
Strucute = 3d world recovery
 
motion means  the location of the camera...
 
he will be doing the grad course on CV ONLINE sometime in the future...
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 </p>