Knowledge-based systems, Overview, SOAR, ARCHITECTURE

Architecture

Processing cycle – decision procedure

Soar’s main processing cycle arises from the interaction between procedural memory (its knowledge about how to do things) and working memory (its representation of the current situation) to support the selection and application of operators. Information in working memory is represented as a symbolic graph structure, rooted in a state. The knowledge in procedural memory is represented as if-then rules (sets of conditions and actions), that are continually matched against the contents of working memory. When the conditions of a rule matches structures in working memory, it fires and performs its actions. This combination of rules and working memory is also called a production system. In contrast to most production systems, in Soar, all rules that match, fire in parallel.

Instead of having the selection of a single rule being the crux of decision making, Soar’s decision making occurs through the selection and applications of operators, that are proposed, evaluated, and applied by rules. An operator is proposed by rules that test the current state and create a representation of the operator in working memory as well as an acceptable preference, which indicates that the operator should be considered for selection and application. Additional rules match with the proposed operator and create additional preferences that compare and evaluate it against other proposed operators. The preferences are analyzed by a decision procedure, which selects the preferred operator and installs it as the current operator in working memory. Rules that match the current operator then fire to apply it and make changes to working memory. The changes to working memory can be simple inferences, queries for retrieval from Soar’s long-term semantic or episodic memories, commands to the motor system to perform actions in an environment, or interactions with the Spatial Visual System (SVS), which is working memory’s interface to perception. These changes to working memory lead to new operators being proposed and evaluated, followed by the selection of one and its application.

Reinforcement learning

Soar supports reinforcement learning, which tunes the values of rules that create numeric preferences for evaluating operators, based on reward. To provide maximal flexibility, there is a structure in working memory where reward is created.

Impasses, substates, and chunking

If the preferences for the operators are insufficient to specify the selection of a single operator, or there are insufficient rules to apply an operator, an impasse arises. In response to an impasse, a substate is created in working memory, with the goal being to resolve the impasse. Additional procedural knowledge can then propose and select operators in the substate to gain more knowledge, and either create preferences in the original state or modify that state so the impasse is resolved. Substates provide a means for on-demand complex reasoning, including hierarchical task decomposition, planning, and access to the declarative long-term memories. Once the impasse is resolved, all of the structures in the substate are removed except for any results. Soar’s chunking mechanism compiles the processing in the substate which led to results into rules. In the future, the learned rules automatically fire in similar situations so that no impasse arises, incrementally converting complex reasoning into automatic/reactive processing. Recently, the overall Universal Subgoaling procedure has been extended through a mechanism of goal-directed and automatic knowledge base augmentation that allows to solve an impasse by recombining, in an innovative and problem-oriented way, the knowledge possessed by a Soar agent.^[3]

Symbolic input and output

Symbolic input and output occurs through working memory structures attached to the top state called the input-link and the output-link. If structures are created on the output-link in working memory, these are translated into commands for external actions (e.g., motor control).

Spatial visual system and mental imagery

To support interaction with vision systems and non-symbolic reasoning, Soar has its Spatial Visual System (SVS). SVS internally represents the world as a scene graph, a collection of objects and component subobjects each with spatial properties such as shape, location, pose, relative position, and scale. A Soar agent using SVS can create filters to automatically extract features and relations from its scene graph, which are then added to working memory. In addition, a Soar agent can add structures to SVS and use it for mental imagery. For example, an agent can create a hypothetical object in SVS at a given location and query to see if it collides with any perceived objects.

Semantic memory

Semantic Memory (SMEM) in Soar is designed to be a very large long-term memory of fact-like structures. Data in SMEM is represented as directed cyclic graphs. Structures can be stored or retrieved by rules that create commands in a reserved area of working memory. Retrieved structures are added to working memory.

SMEM structures have activation values that represent the frequency or recency of usage of each memory, implementing the base-level activation scheme originally developed for ACT-R. During retrieval, the structure in SMEM that matches the query and has the highest activation is retrieved. Soar also supports spreading activation, where activation spreads from SMEM structures that have been retrieved into working memory to other long-term memories that they are linked to.^[4] These memories in turn spread activation to their neighbor memories, with some decay. Spreading activation is a mechanism for allowing the current context to influence retrievals from semantic memory.

Episodic memory

Episodic Memory (EPMEM) automatically records snapshots of working memory in a temporal stream. Prior episodes can be retrieved into working memory through query. Once an episode has been retrieved, the next (or previous) episode can then be retrieved. An agent may employ EPMEM to sequentially play through episodes from its past (allowing it to predict the effects of actions), retrieve specific memories, or query for episodes possessing certain memory structures.

Learning

Each of Soar’s long-term memories have associated online learning mechanisms that create new structures or modify metadata based on an agent’s experience. For example, Soar learns new rules for procedural memory through a process called chunking and uses reinforcement learning to tune rules involved in the selection of operators.