Bostrom 2012 SuperIntelligent Will Motivation
<p>Nov 01, 2012 11:25 PM
 
Nick Bostrom 2012
 
The SuperIntelligent Will: Motivation and Instrumental Rationality in Advanced Artif Agents
 
Future of Humanity Instit.  2012  Oxford...
 
Orthogonality thesis:  intel and final goals (purposes are orthogonal.
 
Instrumental convergence thesis:  will pursue similar intermediary goals.
 
all human minds ... a small, tight cluster.
 
Hannah Arendt and Benny Hill.
neural archit... identical.
 
Orthogonality Thesis:  intel and motivation are orthogonal and can freely vary.
 
David Hume : belief and motivation are separate.
 
arbitary high intel does not entail the acquistion of any such beliefs as are motivating on their own.
 
 
arbitar high intel but constitution so alien as to contain no function analogies to beliefs or desires.
 
 
1.3  Bizarre goals/motivations:  sand-grain counting or paperclip-maximizing.
 
synthetic minds can have utterly non-anthropomorphic goals;
 
1)  stabel pursuit of goals... it will simply get better at pursuing those goals.
 
2)  predictability thru inheritance:  yes... but could be corrupted.
 
3) instrumental convergence
 
2.1  Self-preservation:   longevity helps in achieving goals
 
2.2  goal-content integrity -
software agents easily switch bodies and create exact duplicats, might swap memories,
download skills and radically modify thier cognit architecture and personalities.
might be better individuated as telological threads.
 
2.3 creative enhancement:  
an agent that has access to reliable expert advice may have little need for its own intel/ know.
 
 
suppose an agent is in a position to become the first superintel and thereby obtain a deisive advantage
enabling the agent to shape the future of Earth-origninate life and cosmic resources.
 
2.4
 
more efficient woys of transforming a set of inputs into valued outputs.
 
improved engineerring,  running more quickly, reliably... cheaper and with less energy.
 
technology must be evaluated with ref to some agents final values.
 
von Neumann probes:  automatic, self-meding, self-replicating spaceships.
 
2.5  Resource acquisiton:  tech and resources facilitate physic construciton projects...
 
value of resources dpends on uses to which they can be put...
 
time, space, matter, energy  can be put to almost any goal.  cud be convered to life
run the superintel at greater speed, longer duariont,  or create physical or simulated lives and 
civilizaitons.  backups, perimeter defenses,  enhanced secutiry.
 
Once von Neumann probes can be built... a large portion of the observ. universe cud be grdually colonized.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 </p>