Part 2: Assessing Assessments

Outline

In this part, we explain a key theoretical framework through which we can understand learning outcomes: Bloom’s Revised Taxonomy. Understanding the taxonomy will:

  1. Help us to better analyse the demands of different learning outcomes and
  2. Ask us to think deeply about what it means to be a chemist (at least by degree).

Bloom's Revised Taxonomy

We can think of Bloom’s Revised Taxonomy as an organising framework that can help educators make sense of standards and outcomes based on their environment. The idea is that it stratifies learning objectives by complexity, which then allows us to design assessments that will actually treat an outcome at the required level.

We’re going to divide concepts into two dimensions: the knowledge dimension and the cognitive process dimension. Each dimension features several subcategories, which increase in complexity. Within the knowledge dimension, this means moving from concrete to abstract concepts, from factual to metacognitive knowledge.

Table: the Knowledge Dimension of Bloom's Revised Taxonomy.

Concrete     Abstract
Factual Conceptual Procedural Metacognitive

Essential facts, terminology, or details students must know or be familiar with in order to understand a discipline or solve a problem in it.

a. Knowledge of terminology
b. Knowledge of specific details and elements

Classifications, principles, generalizations, theories, models, or structures relevant to a particular disciplinary area and the relationships between and among them.

a. Knowledge of classifications and categories
b. Knowledge of principles and generalizations
c. Knowledge of theories, models, and structures

Skills, algorithms, techniques and methods, and criteria used to determined when these can be used.

a. Knowledge of subject-specific skills and algorithms
b. Knowledge of subject-specific techniques and methods
c. Knowledge of criteria for determining when to use appropriate procedures

Strategic or reflective knowledge about how to go about solving problems, cognitive tasks and knowledge of self

a. Strategic knowledge
b. Knowledge about cognitive tasks, including appropriate contextual and conditional knowledge
c. Self-knowledge

Within the cognitive process dimension, we consider thinking skills—from lower order to higher order.

Table: the Cognitive process dimension of Bloom’s Revised Taxonomy.

Lower-order         Higher-order
Remember Understand Apply Analyse Evaluate Create

Retrieve relevant information from long-term memory.

a. Recognise
b. Recall

Construct meaning from instructional messages (including written, oral and graphic).

a. Interpret
b. Exemplify
c. Classify
d. Summarise
e. Infer
f. Compare
g. Explain

Use procedures to perform exercises (familiar context) or solve problems (unfamiliar context).

a. Execute
b. Implement

Breaking material into its constituent parts and determining how the parts are related to one another and to an overall structure.

a. Differentiate
b. Organise
c. Attribute

Make judgements based on criteria and standards.

a. Check
b. Critique

Put elements together to form a coherent whole; reorganise elements into a new pattern or structure.

a. Generate
b. Plan
c. Produce

It’s important to acknowledge that these steps are divides in a continuum, and sometimes the lines can be a little blurry between categories and complexities! This isn’t meant as an absolute description, but as a reference point—a springboard for constructive thinking.

The two dimensions interact to comprehensively define learning outcomes: we can think of them as two axes along a plane, with outcomes of different complexities lying at different points on the plane. In particular, as we increase complexity along either dimension, we move from outcomes that describe qualities we may expect of an earlier stage student to those that apply to graduates.

The dimensions of Bloom’s Revised Taxonomy as a comprehensive framework for understanding learning outcomes.

  Factual Conceptual Procedural Metacognitive
Remember List Recognise Recall Identify
Understand Summarise Classify Clarify Predict
Apply Respond Provide Carry out Use
Analyse Select Differentiate Integrate Deconstruct
Evaluate Select Determine Judge Reflect
Create Generate Assemble Design Create

That doesn’t quite cover it, though—this is about more than just ranking learning outcome buzzwords and creating relevant assessments. In our case, Bloom’s Revised Taxonomy asks us to think deeply about what it means to be a chemist-by-degree. What kinds of knowledge do chemists actually need? Does your answer vary across disciplines within chemistry? What about along the cognitive process dimension—are there specific thinking skills that chemists need to exercise more than others? For every knowledge expectation in chemistry, is there an optimal cognitive process level?

 
 

 

 
 

Recall that the key idea is that of the shift in our understanding of learning outcomes, from knowledge, skills, and attributes that graduates should show to ones they will show. So far, we’ve talked about:

  1. Why we assess;
  2. What learning outcomes are;
  3. Complexity in learning outcomes (Bloom’s Revised Taxonomy).

Now we’ve come to the crux of the workshop: how do we check that a particular assessment task actually assesses the learning outcome(s) it says it does? Our answer: the assessing assessments tool!

Part 3 of this workshop explains the tool and gives a couple of examples of how it's used. You can also go back to Part 1 if you want.