DileepThesis Hierarchical Temporal Model
<p>Apr 27, 2009 9:33 PM
 
The No Free Lunch (NFL)   from page 2 
theorem, a fundamental result on search and optimization, addresses aspects about
the complexity of learning [50]. Intuitively, the NFL theorem for learning argues
that no learning algorithm has an inherent superiority over another algorithm for all
learning problems. If an algorithm is superior for a particular problem, it is only
because that algorithm exploited assumptions that are suitable for that problem.
The same algorithm will not do well on a problem that has assumptions dierent
from that of the original one. This means that, to make the algorithms eective, the
machine-learning researcher has to encode knowledge about the problem domain into
the initial model structure. The more prior knowledge that is put in, the easier it is
to learn.
 
From the NFL theorem we conclude that a universal theory of learning is, in  (from page 4)
essence, a universal theory of assumptions. The set of assumptions that a learner
uses to predict outputs, given inputs that it has not encountered, is known as the
inductive bias of that learning algorithm [66, 36]. The more assumptions we make,
the easier it becomes to learn. However, the more assumptions we make, the fewer
the number of problems we can solve. If we need to design an algorithm that can
be applied to a large class of problems, the question we need to ask is: What is
the basic set of assumptions that are speci c enough to make learning feasible in a
reasonable amount of time while being general enough to be applicable to a large class
of problems? We are in luck if the same set of assumptions works for a large class of
problems.
 
There are many anatomical and physiological
<b>details in the brain; how is one to know what is important and what is not? What is a
mere implementation detail that biology has to employ because it has only neurons to
work with, and what is an important computational principle that cannot be missed</b>?
 
Chap 2: Concept of MANIFOLD is essential....  n-dimensional manifold ...
one N vector corresponds to one image of an object  (from whether each of N
receptors in the retina is either firing or not).
If you join all those N-vectors, it produces an n-dimens manifold which
IS the visual concept of that object.
Because of the continuity of motion (locality and inertia of motion)... points that
are close in time (as the viewr moves) are also close on the manifold.
 
HIERARCHY is helpful cuz most real world objects share the same building blocks.
Can learn the aggregate objects by virtue of already knowing the simple objects.
 
 
 
 
 
 
 
 
 
 </p>