DREXLER, Eric, Superintelligence, CSD, July 27, 2016
<p class="MsoNormal" >DREXLER, Eric, Superintelligence, CSD, July 27, 2016</p>
<p>
(student to my R using <b>LYX </b>... a GUI for LaTex)</p>
<p>
he coined the term molec nanotech... Chm of his comm was Marvin Minsky.</p>
<p>
UG and PhD at MIT.</p>
<p>
</p>
<p>
now a researcher with Nick Bostrom at Oxford.</p>
<p>
Elon Musk: with AI we are summoning the demon... (and other quotes)</p>
<p>
Nick Bostrom: book Superintell.</p>
<p>
TOPICS for today:</p>
<p>
disaggreg. the concept of intellig.</p>
<p>
threat models, interventions, and alternatives</p>
<p>
superintell engr...</p>
<p>
safe recursive improvement etc.</p>
<p>
*</p>
<p>
disaggreg the concept of intellig.</p>
<p>
IJ Good defn: 1965</p>
<p>
but a child is intellig cuz of learning capacity and despite lack of competence.</p>
<p>
and can imagine an expert that is v intellig but has a learning disability.</p>
<p>
so, superintellig can consist of focused supercompetences.</p>
<p>
Autogenous AI risks... might be a large attractor zone (ie, the risk region might be large)</p>
<p>
so what is the scope of the safe territory (that is outside the basin of attraction)</p>
<p>
human level AI is sufficient but far from necessary. (setting the bar too high)</p>
<p>
want to block Willful superintell agents from becoming existential threats.</p>
<p>
but might be easier to block broad superintell. from becoming Willful superintell agents.</p>
<p>
*</p>
<p>
superintell. agents... eg if it could design Apollo Mission.</p>
<p>
look at what is required for Apollo: millions of components and thousands of specialties.</p>
<p>
rocket: propulsion engr, struc engr, etc.</p>
<p>
fluid dynamics, polymers, materials processing,</p>
<p>
communicating experts (say, rather than one unitary system)</p>
<p>
off-task human competences:</p>
<p>
tile all the required competences with experts...</p>
<p>
</p>
<p>
a modular AI architecture for interactive engr...</p>
<p>
discuss task specialties...</p>
<p>
generate candidate designs and</p>
<p>
test designs in simulation...</p>
<p>
assess and design performance</p>
<p>
output and explain designs</p>
<p>
remember improved designs...</p>
<p>
this would be an engr design , chat and sketch bot that would interact with users.</p>
<p>
that breaks down specialty generator and explainer</p>
<p>
and design generator and design tester.</p>
<p>
and perhaps a simulation server</p>
<p>
and a design searcher to search a design catalog...</p>
<p>
then how to do recursive improvement...</p>
<p>
general design methods lead to target systems: self-driving cars, med advisor etc.</p>
<p>
incremental automation of AI development pipelines...</p>
<p>
end-game applications (like nuclear weapons)</p>
<p>
but this system is still not an autogenous system with autogen AI risks...</p>
<p>
he calls these abusive applications...(like nuclear weapon design or bioweapon design)</p>
<p>
*
afterward: Eric and I discuss longevity research (another interest of his)
draws my attention to the ITP program at NIA and their successful trials of
rapamycin and also takes metformin and acarbose;
see private notes.</p>
<p>
</p>