Simplicity versus likelihood in visual perception from surprisals to precisals van der Helm 2000

Jun 30, 2011 07:33 PM  30 page paper...  this is an EXCELLENT TUTORIAL on

SIT vs AIT... the best I've found.  a beautiful reconciliation/ unification of the two.

Peter van der HELM  2000  from Psych Bulletin

only two pages of this paper are available free and online if you search the name  BUT...

No actually entire paper is available on Peter's website... see his pubs.  this is an important paper.

The likelihood principle states that the visual system prefers the most likely interpretation of a stimulus,

whereas the simplicity principle states that it prefers the most simple interpretation. This study investi-

gates how close these seemingly very different principles are by combining findings from classical,

algorithmic, and structural information theory. It is argued that, in visual perception, the two principles

are perhaps very different with respect to the viewpoint-independent aspects of perception but probably

very close with respect to the viewpoint-dependent aspects which, moreover, seem decisive in everyday

perception. This implies that either principle may have guided the evolution of visual systems and that

the simplicity paradigm may provide perception models with the necessary quantitative specifications of

the often plausible but also intuitive ideas provided by the likelihood paradigm.

Look at figure 1...  great example:  very clear preference for the black diamond occluding

the shape in part b.  why that preference?  is it that ...

the LIKELIHOOD PRINCIPLE:   visual system prefers the most likely interpretation... highest prob. of being correct

vs  the SIMPLICITY PRINCIPLE:  the simplest  (ie the SHORTEST DESCRIPTION).

deep roots in history of science.

Wm of OCCAM  1290 to 1349...  the most simple interpretation of the data is the best.

MACH  suggested that simplicity and likelihood might be difft sides of the same coin.

empirically:  concrete applications of simplicity and likelihood yield the SAME prediction.

debates in 1980s and 1990s involved  Leeuwenberg,  van der Helm Biederman etc.

See figure one:  likelihood acct is in terms of positional coincidences and is a probabilistic

view-dependent acct.

In contrast, the simplicity acct is in terms of shape complexities (describes the shapes and is

apt to be viewpoint independent.

In this paper he shows how they can be compared theoretically.

His paper is based on AIT  = algorithmic information theory.

page 772

von Helmholtz  "  we perceive the most likely objects or events that would fit the sensory pattern that we are trying to interpret"

Perception is thus conceived as a neural activity the outcome of which can be modeled by a process of

hypothesis testing.

In other words, perception is modeled as selecting, for a given proximal stimulus,

the interpretation with the highest probability of being correct.

The likelihood principle corresponds to the assumption that perception is as veridical as it can be.

Viewpoint-independent aspects and prior probabilities:

traditional idea:  prob of an interpretation is higher if the interp reflects 

a more frequent situation in the world

OR  with more bio importance as reflected by phylogenetic acquired knowledge.

If these are present in the real world, then they are viewpoint-independent probs = prior probabilities (of objects themselves

as opposed to the views they present).

Viewpoint-dependent aspects and conditional probs  (eg in Biederman's work) propose heuristics for inferring

interpretations given proximal stimuli (the actual views).

Many of these heuristics are the Gestalt laws of perceptual organization...  Kohler 1920 and Wertheimer 1923.

proximity, symmetry, similarity, good continuation.

Rock 1983:  avoidance-of-coincidence heuristic,  which excludes interps that imply an accidental viewpoint.

also argued by Tom Binford,  Marty Tenenbaum Irv Biederman...  avoid accidental viewpoints.

The assumption is of a GENERAL VIEWPOINT (not special).

needle example:  needle is always a line segment except 

when viewed at the ends where it collapses to a single point.

The conditional prob. of an interp is the prob that the given proximal stim (the view) 

arises conditional on the real distal stim (the actual object as hypothesized.)

For instance, a distal straight edge yields proximally a straight line

from almost any viewpoint position, so that the hypothesis that a

proximal straight line is caused by a distal straight edge has a very

high conditional probability...

Figure two is also quite good  rectangle and triangle..

A is the most general ... separate... F is the least general... forming a single figure..... attached.

page 774  he uses BAYES RULE.  provides AIDS example.

SIMPLICITY PRINCIPLE

Koffa named the law of PRAGNANZ (pithiness)

pragnanz = stability, harmony, simplicity.

PERCEPTION is a neural system that when presented with a stim, is like other physical systems,

it settles into an equilibrium involving minimum energy load. (prob. from Helmholtz)

The resulting neural pattern of activation then

forms the mental repres of the stimulus interpretation.

page 775:  key discussion:

this led Julian Hochberg and McAlister 1953 to propose the simplicity principle:

the less amt of info is needed to define an organization,

the more likely that the figure will be so perceived.

the number of difft items we must be given to specify a pattern.

Attneave 1954 ... also discussed this.

Simplicity:  coding languages:   implemented in various perceptual coding languages:

page 776  Hierarchical structure of descriptive codes:  (see Josh Tenenbaum videos and work)

allows for simult description of difft features not only at a same hierarch level but also at difft levels.

circular pattern built up of small squares OR

of a square built up of small circles.

Figure 3:  another great figure:  what is it?  how is it built up?

The 4 triangles pop out immediately into the foreground for me (enclsoing a diamond)

I must be prompted for the big triangle formulation and 

absolutely do NOT see the intersecting parallelograms without considerably prompting and work.

(To me it seems to be either 4 triangles enclosing a diamond or 

two intersecting big triangles (one inverted)  (It's really hard to see

the version consisting of two parallelograms.  The triangles are SO compelling!  Why???)

Neisser 1967 was also interested in this.

page 776 col 2:  what are the primitives in perception?

r they pixels, dots, edges, blobs?

Biederman 1987 proposed the geons...  simple volumes.

Within the simplicity paradigm the idea is that perception starts by considering all possible primitives,

and uses those with the simplest descriptive code for the stimulus at hand.

page 777:  simplicity (to Hochberg and McAlister 1953)  means "the number of items we must be given."

The foregoing suggests that the simplicity paradigm can be seen

as a scientific enterprise that, at one stroke, might solve the two

questions that remained pending for the likelihood paradigm. That

is, first, descriptive codes might specify the perceptual categories

to which probabilities are to be assigned, and, second, the complexities

of these codes might be used to quantify these probabilities.

Thus, the simplicity paradigm might provide a quantitative

specification of the intuitive notions provided by the likelihood

paradigm. This is the enterprise Hochberg and McAlister (1953)

more or less proposed and that has been explicitly promoted by

Leeuwenberg and Boselie (1988). By the way, once the perceptual

categories have been specified by means of descriptive codes, one

could of course still decide to search for their real-world probabilities.

However, descriptive codes, regularity, and simplicity

form a strong trinity, so that it seems expedient to at least consider

the option of using complexities to quantify probabilities.

This option might be fruitful especially in situations like perception

where one does not have direct access to the real probabilities. In

mathematics, Solomonoff (1964a, 1964b) explicitly had such situations

in mind when he proposed to undertake the very same

scientific enterprise (see the fourth part, The Relation Between

Simplicity and Likelihood).   <<<  note:  this is exactly what Josh Tenebaum is doing: see his videos and work>>

Solomonoff (1964a, 1964b) not only was interested in the possibility

of a useful relation between complexities and probabilities

(rather than in the categorization issue) but also solved a problem

that seemed to nip the entire enterprise in the bud. 

That is, if one defines the complexity of an object by the length of its shortest

description, then different descriptive coding languages generally

yield different complexities for one and the same object. Hence, it

seems impossible to speak of "the" complexity of an object. Thus,

the question is: Which descriptive coding language should be

used? Solomonoff, however, showed that it does not matter much

which descriptive coding language is used because the complexity

of an object cannot vary arbitrarily over descriptive coding

languages.

Solomonoff 's finding implies that different descriptive coding languages yield object complexities

that are fairly close to one another.

In other words, it implies...  SIMPLICITY is a fairly stable concept and it is possible to speak of

THE COMPLEXITY OF AN OBJECT !!!

Solomonoff's finding, known as the lnvariance Theorem,

marked the beginning of AIT as a separate mathematical research

area. Independently, this theorem has been proved too by the other

two founding fathers of AIT, Kolmogorov (1965) and Chaitin

(1969). Also independently, Simon (1972) observed this invariance

in perception. Simon compared six perceptual coding languages

from the literature and found that their quantifications of

complexity are highly correlated. Simon's finding can be seen as

an empirical proof that the Invariance Theorem is valid in perception.

Simon 

Leeuwenberg's coding model 1969 and 1971 became known as SIT Struc Info Theroy.

SIT is nowadays the most elaborated coding approach to perception.

SIT conceives of perception as a neural activity the outcome of which can be modeled by a process

of hypthesis testing - but now selecting the most simple hypothesis.

SIT started out as viewpoint independ  and only later consider viewpoint-dependent aspects...

but now integrates both.

page 778

SIMPLER CODES WIN !  (and dictate simpler perceptions)

great example:  figure 4   most general view is the most probable

the fourth version is the one with the MOST coincidences and the LEAST probable

(and therefore the least readily seen).

page 779:  Integrating Prior and conditional complexities:

just sum them...  this turns out to explain 95% of the empirical data from the literature.

This integration was developed in parallel and indep. in AIT.

Rissanen 1978 proposed the MDL = minimum description length principle:

The best hypothesis to explain given data is the one that minimizes the sum of

(a) the information needed to describe the hypothesis; and

(b) the information needed to describe the data with the help of the hypothesis.

Hence, in this definition, the first amount of information is the

prior complexity (given by the shortest description of the hypothesis

as such), and the second amount of information is the conditional

complexity (given by the shortest description of the data,

starting from the hypothesis).  MDL = MINIMUM DESCRIPTION LENGTH !!!

Various incarnations of the MDL principle are nowadays being

used for prediction in a wide range of statistical problems, among

others in molecular biology (see Li & Vit_ayi, 1997, p. 376) and

cognitive science (see, e.g., Myung, 2000; Myung & Pitt, 1997).

The MDL principle can in fact be seen as a modern version of OCCAM's razor...

column 2 of page 779 is dynamite... just read it !!!

the MDL principle in AIT complies with van Lier's integrative approach to perception.

It gives a formal characterization of the simplicity principle

by formula (2)  ... visual system selects the hypothesis H which minimizes I (H|D)

= posterior complexity of interpretation H given proximal stim D  and  that this

complexity is given by  the SUM of the

viewpoint-independent prior complexity I(H) of the distal shape  +

the viewpoint-dependent conditional complexity I(D|H).

also see the examples in figure 5 and figure 1 again.

page 780  again  Figure 5 is absolutely great!!!

Figure 5B  suggests a weird, irregular octagon (to avoid the coincidental positioning of the rectangle..

p 779 DISCUSSION

above minimal information only describes the outcome.... the perception itself

NOT he neural process that led to it... which remains  AUTOMAGIC  ie  automatic and mysterious.

Indeed, both principles conceive of perception as a neural activity the outcome of which can be modeled by a process of hypothesis testing

eventually selecting the most simple or most liely hypothesis.

how does it work???

page 780

this is the subject of PROCESS MODELS of PERCEPTION

page 781  The Age of Information

Simplicity versus likelihood in visual perception from surprisals to precisals van der Helm 2000

Attachments