Part 2: Assessing Assessments
Outline
In this part, we explain a key theoretical framework through which we can understand learning outcomes: Bloom’s Revised Taxonomy. Understanding the taxonomy will:
- Help us to better analyse the demands of different learning outcomes and
- Ask us to think deeply about what it means to be a chemist (at least by degree).
Bloom's Revised Taxonomy
We can think of Bloom’s Revised Taxonomy as an organising framework that can help educators make sense of standards and outcomes based on their environment. The idea is that it stratifies learning objectives by complexity, which then allows us to design assessments that will actually treat an outcome at the required level.
We’re going to divide concepts into two dimensions: the knowledge dimension and the cognitive process dimension. Each dimension features several subcategories, which increase in complexity. Within the knowledge dimension, this means moving from concrete to abstract concepts, from factual to metacognitive knowledge.
Table: the Knowledge Dimension of Bloom's Revised Taxonomy.
Concrete | Abstract | ||
---|---|---|---|
Factual | Conceptual | Procedural | Metacognitive |
Essential facts, terminology, or details students must know or be familiar with in order to understand a discipline or solve a problem in it. a. Knowledge of terminology |
Classifications, principles, generalizations, theories, models, or structures relevant to a particular disciplinary area and the relationships between and among them. a. Knowledge of classifications and categories |
Skills, algorithms, techniques and methods, and criteria used to determined when these can be used. a. Knowledge of subject-specific skills and algorithms |
Strategic or reflective knowledge about how to go about solving problems, cognitive tasks and knowledge of self a. Strategic knowledge |
Within the cognitive process dimension, we consider thinking skills—from lower order to higher order.
Table: the Cognitive process dimension of Bloom’s Revised Taxonomy.
Lower-order | Higher-order | ||||
---|---|---|---|---|---|
Remember | Understand | Apply | Analyse | Evaluate | Create |
Retrieve relevant information from long-term memory. a. Recognise |
Construct meaning from instructional messages (including written, oral and graphic). a. Interpret |
Use procedures to perform exercises (familiar context) or solve problems (unfamiliar context). a. Execute |
Breaking material into its constituent parts and determining how the parts are related to one another and to an overall structure. a. Differentiate |
Make judgements based on criteria and standards. a. Check |
Put elements together to form a coherent whole; reorganise elements into a new pattern or structure. a. Generate |
It’s important to acknowledge that these steps are divides in a continuum, and sometimes the lines can be a little blurry between categories and complexities! This isn’t meant as an absolute description, but as a reference point—a springboard for constructive thinking.
The two dimensions interact to comprehensively define learning outcomes: we can think of them as two axes along a plane, with outcomes of different complexities lying at different points on the plane. In particular, as we increase complexity along either dimension, we move from outcomes that describe qualities we may expect of an earlier stage student to those that apply to graduates.
The dimensions of Bloom’s Revised Taxonomy as a comprehensive framework for understanding learning outcomes.
Factual | Conceptual | Procedural | Metacognitive | |
Remember | List | Recognise | Recall | Identify |
Understand | Summarise | Classify | Clarify | Predict |
Apply | Respond | Provide | Carry out | Use |
Analyse | Select | Differentiate | Integrate | Deconstruct |
Evaluate | Select | Determine | Judge | Reflect |
Create | Generate | Assemble | Design | Create |
That doesn’t quite cover it, though—this is about more than just ranking learning outcome buzzwords and creating relevant assessments. In our case, Bloom’s Revised Taxonomy asks us to think deeply about what it means to be a chemist-by-degree. What kinds of knowledge do chemists actually need? Does your answer vary across disciplines within chemistry? What about along the cognitive process dimension—are there specific thinking skills that chemists need to exercise more than others? For every knowledge expectation in chemistry, is there an optimal cognitive process level?
Recall that the key idea is that of the shift in our understanding of learning outcomes, from knowledge, skills, and attributes that graduates should show to ones they will show. So far, we’ve talked about:
- Why we assess;
- What learning outcomes are;
- Complexity in learning outcomes (Bloom’s Revised Taxonomy).
Now we’ve come to the crux of the workshop: how do we check that a particular assessment task actually assesses the learning outcome(s) it says it does? Our answer: the assessing assessments tool!
Part 3 of this workshop explains the tool and gives a couple of examples of how it's used. You can also go back to Part 1 if you want.