EHLEN, Patrick, Doing Semantics with GPUs, Loop AI Labs, Oct 7, 2015
<p class="MsoNormal" >EHLEN, Patrick, Doing Semantics with GPUs, Loop AI Labs, Oct 7, 2015</p>
<p>
This is a seminar in the NVIDIA Symposium today at Stanford in the Jen-Hsun Huang Bldg.</p>
<p>
mostly on language understanding.</p>
<p>
a little abt GPUs and a lot abt semantics today.</p>
<p>
IDC forecast <b>12.5 B market in 2019 CAGR at 35%</b></p>
<p>
which is more difficult... vision or doing language.</p>
<p>
<b>is language solved? NO.</b></p>
<p>
<b>how many creatures can see? now, how many can talk? one!</b></p>
<p>
can communicate with gestures.</p>
<p>
embed and recurse over long strings.</p>
<p>
<b>infinite use of finite means...</b></p>
<p>
<b>This is the rat that ate the malt that... that lived in the house that Jack built.</b></p>
<p>
<b>only showed up abt 50K to 200K years ago;</b></p>
<p>
<b>3 possible sources</b>...</p>
<p>
<b>interface (attention/ ST memory or vocal tract)</b>
<b>or learning algo </b>
<b>or brain architecture.</b></p>
<p>
does not seem like interface (similar to primates)
poss not learning algo... also similar.
<b>but maybe brain architecture... DL = deep learning.</b></p>
<p>
Geoff Hinton 81 and 86:
John hit Mary.
agent relation and "patient"</p>
<p>
is it just assemblies.</p>
<p>
Frankland and Greene 2015.. in PNAS 112:37</p>
<p>
An architecture for encoding sentence meanings in left mid-superior temp cortex... GET THIS!
he urges for map to units
or to matrix or vectors like human, young, animal.</p>
<p>
finds these in IT.</p>
<p>
<b>Recap: human capacity arises from architecture.</b></p>
<p>
Semantics... what does this mean.</p>
<p>
dictionary: eg bachelor #1... guy with a rose...</p>
<p>
<b>how about a seal who is young and unmated.</b></p>
<p>
look it up in a dictionary...</p>
<p>
map into directed graph...</p>
<p>
human, male , never married etc. in hierarchy.</p>
<p>
BUT see Lakoff and Joshnson or Hofstadter...</p>
<p>
robin is more repr of bird than penguin.</p>
<p>
we do everything by analogy.</p>
<p>
edge cases... <b>is a widow a woman who murdered her husband?</b></p>
<p>
<b>Max went too far and teapotted a policeman</b> (from Herb Clark)</p>
<p>
context words (like the quick, brown fox) the quick, brown.</p>
<p>
young gets distributed among several latent features.</p>
<p>
<b>DEEP LEARNING</b></p>
<p>
new way to learn distrib repr using neural networks..</p>
<p>
learn data as it comes in...</p>
<p>
and learn from sequences</p>
<p>
visual features CNNs
parse structure recursive NNs (feed in parse tree.</p>
<p>
needs lots of matrix multiplication... made faster by GPUs.</p>
<p>
to do human capacity cognition need</p>
<p>
ability to handle context feature input</p>
<p>
LOOP AI
we have GPU based appliance...</p>
<p>
learn sfrom unstruc text data => strucutred repres.</p>
<p>
Q experience with mixed modalities? A working on that...
</p>