Lecture 10: Reinforcement Learning
<p>May 15, 2015 06:05 PM</p>
<p>
this is Pieeter Abbeel at Cal. CS188;</p>
<p>
how do people and animals learn by reward...</p>
<p>
Agents and Enviro (same as MDP... Markov Decision Proc)</p>
<p>
RL slide</p>
<p>
agent gets feedback in form of reward and ACT so as to maximize rewards.</p>
<p>
all learning base don samples of outcomes</p>
<p>
Agent: Enviro in state S; agent takes action A and gets reward r...</p>
<p>
what is the right thing to do in each state.</p>
<p>
you learn as you go along;</p>
<p>
shows Aibo learning to run</p>
<p>
shows robotic snake (from Andrew Ng) learning to mount step and to side-wind on step.</p>
<p>
toddler robot (auto walks downhill; swinging legs)
motors in ankles... (Seung robot from 2005)
TD learning... figures how to walk (MIT)</p>
<p>
CRAWLER (student project)...</p>
<p>
hip angle and knee angle pulling a block...</p>
<p>
running Q learning...
- reward to move to the right...
it gradually learns (it does not know that it's a crawling bot)</p>
<p>
at 10:49 RL still assume an MDP...</p>
<p>
a set of states; a set of actions per state
a model R(s, action, and state prime)
a reward function R</p>
<p>
and T a transition model</p>
<p>
looking for a policy Pi(s)</p>
<p>
see car (go as fast as poss w/o getting overheated);</p>
<p>
context is still MDPs..</p>
<p>
fast and a slow action...</p>
<p>
as an RL learner you don't know that overheated is bad.
must actually try out stuff..</p>
<p>
MDPs are offline!!! (get a model of the world and make a plan then execute it);</p>
<p>
online learning... robot must actually FAIL... don't even know where the failure states are.</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>
<p>
</p>