Bloom’s (Bloom (Ed.), 1956) well known taxonomy identifies six levels of cognitive skills, namely Knowledge, Comprehension, Application, Analysis, Synthesis and Evaluation. Imrie’s RECAP model (Imrie, 1984) adapts Bloom’s taxonomy by dividing cognitive skills into two tiers. The first tier consists of the first three levels of Bloom’s taxonomy namely Knowledge, Comprehension and Application but expresses them as the skills of Recall, Comprehension and Application. Bloom’s final three levels of Analysis, Synthesis and Design are combined into a Problem-solving skills category. Imrie’s simplified taxonomy can be more useful because of the close integration of all these skills in many computer-based problem-solving activities. Problem-solving often requires more than one skill and this broader classification removes the need for unnecessary distinctions in measuring skill performance. Achieving problem solving skills is the ultimate goal. Whilst they can be taught and assessed at an early stage (Cox and Clark, 1998), they are still dependent on the tier 1 skills in a particular subject. When students first come to grips with a subject there is a tendency for knowledge and comprehension questions to dominate quizzes and for problem-solving to dominate later quizzes. However, it is possible for problem-solving questions to be set early in the course (Cox and Clark, 1998). We now discuss and give examples of test questions at different cognitive levels.

Knowledge Questions

In early programming courses, the initial knowledge required is of the syntax of the language and the programming environment. By the end of the subject this knowledge is assumed and it is unlikely to be tested in a final exam. This is not to deprecate knowledge. Other cognitive skills depend on it. The answers to the question “what is the best way to solve this problem”, for example, will depend on the students’ knowledge of class libraries.

One or two simple knowledge questions can be useful in settling students into a test, especially early in the course. Question 13 is an example of a knowledge question. 164 D. Clark

Comprehension

Comprehension needs to followquickly on the heels of knowledge. It consolidates knowledge and allows the internalization of knowledge. The easiest comprehension questions to construct are those where the students are required to trace code and find the final value of a variable. They can be used for selection statements, loops and arrays. Questions 3, 4, 7, 8, 9, 12 and 17 are examples.

 Application

For a programmer, application is the ability to write programs and, at a finer grained level, to write a function or module to perform a specific task. This ability is learned in assignments and can best be tested in a standard written exam. However, “complete the code” multiple choice questions can be devised at this level. Question 10 is one such example.

 Problem Solving

One of the most common tasks in maintenance is to work out what some code is doing. A popular type of question in a final exam is “what does this function do?” A variation is “under which conditions does this function perform correctly?” These tasks require the code to be analysed. Multiple choice questions which test analysis are not as easy to devise as comprehension questions, but with a little thought some of the tracing questions can be recast into analysis questions. Questions 11 and 17 is an example of such a pair. They were used as the first two questions in test 3, where it was hoped that the initial tracing would help the analysis. Question 5 is also an analysis question. Although multiple choice evaluation and synthesis questions can be devised (Cox and Clark, 1998), they are difficult to construct in quantity and there were none in the tests

used in this study.

Reference :

David CLARK, “Testing Programming Skills with Multiple Choice Questions,” Informatics in Education, Vol. 3, No. 2, 161–178, 2004.