This section describes the theory of skill, models for measurement, operationalizations of performance, and instrument validity. Fig. 1 shows how the fundamental concepts involved are related. The skill measure is indicated by the performance of an individual on a set of tasks. Each task is thus an indicator [40], which in turn is defined by a scoring rule that is applied to the time and quality of task solutions (i.e.,a “response” in the figure). The arrows show the direction of reading and causality. The part of the model with arrows pointing downwards constitutes a reflective model. The part with arrows pointing upwards constitutes a formative model [50].

 Theory of Skill

In this work, skill is considered as a specific type of ability, albeit with some distinguishing features.

Generally, all human abilities are “defined in terms of some kind of performance, or potential for performance” [33, p. 4]. “The term ability . . .may refer to measures of . . . an underlying latent variable, which is presumed to be a continuous monotonic increasing function of the observed measures of performance” [60, p. 122]. Thus, skill has—together with concepts such as aptitude, achievement, capacity, competence, expertise, and proficiency—a monotonic

relation to performance.

This positive relation is also an assumption in research on expertise, where reliably superior performance on representative tasks is one of several extended aspects of expertise [55]. According to Ericsson, “[a]s long as experts are given representative tasks that capture essential aspects of their expertise, they can rely on existing skill and will exhibit the same stable performance as they do in everyday life” [52, p. 52]. Unlike some abilities, skill is a psychological variable that can be defined theoretically. Over 80 years ago, Pear [96] recommended using the term for higher levels of performance and then only in conjunction with well-adjusted performance. According to Fitts and Posner [62], the acquisition of skill consists of three overlapping phases. During the initial, cognitive phase, an individual uses controlled processing of

information to acquire facts on how to perform a task successfully. This phase is sometimes referred to as the knowledge acquisition phase, where declarative facts (i.e., knowledge) “concerned with the properties of objects, persons, events and their relationships” [102, p. 88] are acquired. In the second, associative phase, facts and performance components become interconnected and performance improves, with respect to both number of errors and time. In the third, autonomous phase, tasks are accomplished fast and precisely with less need for cognitive control. Although much of the earlier research on skill was conducted on motor skills, Anderson and other researchers devoted much attention to the research on cognitive skills in general during the 1980s (e.g., [4]) and programming skills in particular (e.g., [5], [6], [111]) using Fitts and Postner’s [62] early work. Anderson [5] noted that the errors associated with solving one set of programming problems was the best predictor of the number of errors on other programming problems. We now examine how such insights can be used in the development of a model for measuring skill.

References :

Gunnar R. Bergersen, Dag I. K. Sjoberg, and Tore Dyba, Member, IEEE, “Construction and Validation of an Instrument for Measuring Programming Skill.”