Reading:
LaFrance, M., 1992, Excavation, capture collection and creation: computer scientists’ metaphors for eliciting human expertise, Metaphor and symbolic activity, 7(3 and 4): 135-156. |
psychologists try to develop a vocabulary to describe the human complement of engineers terms like input, output, information channel, channel capacity.
The Magical Number Seven; Plus or Minus Two
processing information in 'chunks'
Plans and the Structure of Behavior
claimed that behaviour is generated by mental processes, rather than environment.
Syntactic Structure
'proved' that natural language competence could not acquired by stimulus-response.
Information Processing Model of Mind
views all mental activities performed by humans as information processing and the human mind as an information processing system - provides the general theoretical framework for cognitive psychologyand cognitive science.
Information comes in through sense receptors, mental operations are applied to and changes it until an output is ready to be stored or used to generate behaviour. The performance of a cognitive task involves a sequence of mental operations (cognitive processes) on mental objects (cognitive structures).
The central hypothesis is that thinking is governed by programs that organize myriad of simple information processes into orderly complex sequences (best understood by the computer analogy). Winograd and Flores summarize this as follows:
Winograd and Flores, 1986: 25
Newell and Simon introduced the concepts of problem space and task environment. The problem space is a person's internal (mental) representation of a problem, and the place where problem-solving activity takes place. The task environment is the physical and social environment in which problem solving takes place. The reason for this distinction is that individual behaviour influences problem solving; this influence is greater the less structured the task is.
Situations which do not influence individual behaviour can be studied by only analysing the task environment. Eg., economic theory is based on the task environment only, assuming that humans are always motivated to maximise their utility; they are expected to behave rationally towards this goal, and everything of importance for problem solving is given by the task environment. Therefore, economics is a science about the task environment.
Where behavioural aspects of problem solving are closely related to the decision maker and not to the task environment, we have to look inside the person's mind to explain this behaviour. Unstructured environments are open for individual behaviour, well-structured environments encourage common behaviour.
Newell and Simon call the model of the task environment a task model(sometimes also known as a competence or epistemological model) and the model of the problem space a performance model:
a task model represents generalized concepts (objects, relations, processes and strategies), and describes a typical high-level problem solving strategy within a domain. It represents an abstract, stereotyped performer within this domain.
a performance model is a model of the problem space and represents the problem solving behaviour of one person who is performing a specific task, but are not adequate for system development since they are constrained to a single performer on a single task.
Both task and performance models are required to enable problem solving behaviour to be adequately modelled within a specific domain.
The problem spaceis seen as consisting of knowledge states, and problem solving proceeds by a selective search within the problem space, according to Newell and Simon using rules of thumb (heuristics) to guide the search.
Newell and Simon's distinction of task and performance models compares and also contrasts with more conventional distinctions of competence and performance introduced to structural linguistics by Chomsky and subsequently widely adopted in social sciences, and also the anthropological terms of emic and etic.
Describes how a problem is solved.
This type of knowledge provides direction on how to do something.
Describes what is known about a problem.
This includes simple statements that are asserted as true or false, but also a list of statements that more fully describes some object or concept.
Describes knowledge about knowledge.
This type of knowledge is used to identify other knowledge that is appropriate for problem solving.
Describes a rule-of-thumb that guides reasoning.
Often called shallow knowledge, it is empirical and represents knowledge gained by experience, but may draw upon fundamental laws, relationships, etc. (sometimes referred to as deep knowledge)
Describes knowledge structures of an experts overall mental model of the problem.
The expert ’s mental model of concepts, subconcepts, and objects is typical of this type of knowledge.
The term 'first generation' connotes obsolescence (Johnson and Tomlinson, 1994), which is unfortunate for many successful systems were developed in this way, and knowledge extraction is probably still the preferred development path, at least for modest systems. Moreover, the `extraction' metaphor is still very much alive in knowledge elicitation insofar as it is construed as a psychological enquiry (e.g. Hoffman et al, 1995), an attitude shared by the second generation, for whom modelling or `knowledge construction' appropriately describes the wider exercise of knowledge acquisition. A summary of the main differences between the two frameworks is shown in Table 1.
| First Generation | Second Generation | |
|---|---|---|
| Perspective on knowledge | Propositional content | Mental-model |
| Method | Empiricism | Iterative modeling |
| Mode of acquisition | Experimental | Analytical |
| Analyst orientation | Observational | Participant-engagement |
| Referent system | Closed | Open |
| Representation | Rule-base | Series of models |
The first generation understood knowledge as propositional content that could be acquired under appropriate experimental conditions. These conditions demanded closure of the system under consideration (the problem-solving system) and a disengaged (value-free) analyst or knowledge-engineer. As expert problem-solving was assumed to be rule-based, and this could be achieved by expert-systems, knowledge acquisition was a transfer process that could be refined in, and validated by, performance.
The second generation assumes expert problem-solving to be model-based, at least with some degree of abstraction, but distinguishes performance (the outward manifestation of the model), from competence (the essence of the model), and arrives at an understanding of knowledge acquisition akin to reverse engineering. Because the underlying (competence) model must be abstracted from a series of performances, the analysis must proceed in a series of interactions in which the analyst plays a crucial intellective role.
Morik (1991) identifies four distinct assumptions about knowledge held by practitioners and researchers that yield different methodological approaches. Her analysis suggests that the transition to modeling frameworks, as a shift in paradigmatic understanding has been far from categorical. By focussing on researchers' ontological understanding of knowledge, rather than the claims made about methods, she demonstrates that many contemporary approaches are marked more by their similarity with the first generation, than by difference.
Table 2 is based on Morik's distinctions, the progression from first to second generation roughly going from left to right. While each perspective attempts to inject some methodological rigour, the original objectives, the transfer or extraction and representation of knowledge remain intact. Note how the formula
model = method specification + domain knowledge
remains faithful to AI axiom that intelligent skills are the product of knowledge and inference, which lay behind the first generation's faith in prototyping with expert system shells, and can be seen to continue in the second generation paradigm.
| Transfer | Performance | Knowledge-level | Constructive | |
|---|---|---|---|---|
| Understanding of knowledge | The basis of expertise, everything an expert uses for problem solving. | The basis of a system, every item an interpreter uses for problem solving. | Whatever can be ascribed to an agent such that its behaviour can be computed according to the principle of rationality. | A constructed description of observations such that the description satisfies the need for explaining the observations. |
| Empirical focus | Facts, rules patterns, heuristics, and operations used by humans to solve problems. | Whatever makes the system solve problems must be knowledge. | Developing operational description in terms of necessary knowledge. | Constructing, applying, revising, and enhancing knowledge. |
| Methodological approach | Elicitation and formalization or coding. Prototyping. | Modular from generic building blocks. | Step-wise refinement: conceptual modeling, specification, knowledge base. | Cyclical, `sloppy-modeling'. |
| Research objective | Reliable methods for experts and analysts to report on problem solving. | Generic structures of problem-solving tasks (e.g. shells). | Formulating a class of problem solving behaviours and domain ontologies. | Developing process models. |
The first generation extraction view of knowledge acquisition had the virtue of a clearly defined psychological object. Expert performance was deemed to result from the application of rules which could be elicited and replicated in a machine. As the second generation phase introduced the idea of modelling as a constructive activity, with the knowledge engineer imposing structure on elicited data, any psychological criteria were left somewhat groundless, begging the question of what it is that is being modeled.
For Clancey (1989), knowledge acquisition is primarily concerned with modeling a system in which the expert is involved, and of which the expert may act as an informant, but the expert and the system are not co-extensive. Crucially, the expert is contained by a wider system which may subsume some of the problem-solving competence attributed (by an observer) to the expert. While Clancey's concept of qualitative process modeling is widely accepted, modeling mental content and processes remain the paramount objective.
Rob Stephens
Room 2P27
Tel: 3136
Robert.Stephens@uwe.ac.uk