NLP Natural Language Thrun Norvig Course
<p>Dec 25, 2011 09:46 PM
 
Peter Norvig lectures...
 
use language sets us apart from all other animals and machines and
want to talk to our computers...
 
want our computers to be smarter...  need to read text and make sense of it...
 
Language Models...
 
1)  sequences of letters  (scrabble letters)   CAB... probabilistic and word-based
learned from data
 
2) trees  like  Sentence =  noun phrase + verb phrase
noun phrase = name = Sam and   VP = verb = slept   SAM SLEPT
this is logical; trees and categories...  deep structure
these are hand-coded... linguists write them down
 
but model 1 is probabilistic and learned...
 
but u cud  go back and forth and have probab trees etc.
 
Honk if you love the bag of words model.... gets idea across   bumpersticker....
 
every word is independent  (first stab at problem)
 
P = prob model  eg  P(w1 w2 w3  wn)   P(w1:n)  prob of the sequence of w1  etc.
 
conditional probability =  PRODUCT of all the sub probs of  w (i-1)  etc.
 
cud make Markov assumption....  depdns only on previous...
 
stationarity...  does not depend on 1st sentence or whether its the Kth sentence  (but does depend
on context of the word)
 
smoothing eg laplace ...
 
saw this in spam filtering model...
 
may want to recover hiddden variables like parts of speech
 
or consider  New York City  as a single phrase...
 
use language for learning....
 
he typed in full moon and ice cream  to Google trends...
 
ice cream... popular in the summer... not so much in the winter...  can get solar year from this...
 
google  icream sandwich... gener a blip ...
 
moon phase... just see users use full moon once a month...
 
N gram models of language...  what r we giving up and what r we getting..
 
unigram....   no adjacency...
 
bigram .... prob of a word given the previous word...
 
hark ye;  if thou;  liberl opposition...
 
trigram  models...
I will never yield;  I woud learn of noble edwward's sons...
 
n=4   four grams....
 
betwixt their two estates...
even to the frozen ridges f the alps...  I know my duty...
capturing some coherency... but not everything....
 
probabilistic letter models:
scrabble letter   XJVY    good for dealing with unique words...
 
Hello, World  or Guten Tag, Welt
Salam Dunya
 
can get 99% accuracy of language id just by looking at letter probs...
 
 
Peope:  Steve  or Bill are common as people
Places  San is common, New is common
Drugs  ending in x is common... starting with z is common  Zyrtec
 
Gzip command:  for classification
 
gzip is a compression algorithm...
eg is_  can be represented as only one byte to compress it
English will have an entry for compresison for is_ ...
 
Segmentation....  how to break it upinto words...
does the first word in chinese correspond to one word or what...
 
choosespain.com   segmentation problem
 
s* = best segmentation =  maxP(w1:n)
can use naive bayes (each word is independent...
 
nowisthetime  eg.... 11 spaces in btwn characters  can either occur or not there,
so answer is   2(n-1)...
 
 
 
 
 
 
 
 </p>