writing rubrics shouldn’t be an imposition

I had an interesting conversation with a couple of colleagues yesterday, concerning the value of rubrics. I write them routinely (must be my background as an examiner at the national level), but my friends really didn’t seem to see the point. ‘You just get a feel for what’s a good essay & a bad one,’ they said, ‘and anyway we don’t have time to write a whole bunch of model answers; it’s quicker just to get in there & start marking. Besides, you can never include every possible answer. ‘ ‘And,’ they said – we were talking about rubrics for someone else to use in marking – ‘it’s far more consistent just to do it all yourself.’

I do agree that some essays spring out as being absolutely wonderful (the very first exam script I marked yesterday was a case in point: a beautifully-constructed answer to a ‘design-a-plant’ question) while occasionally you’ll also come across one that makes you feel like banging your head on the desk. But how can you be sure that you’re treating them consistently? After all, with a big class you’ll likely be marking exam scripts for several days, & your concentration & energy levels are going to vary over that time! Constructing a marking rubric before beginning the marking task will help with that.

It doesn’t have to take a heap of time either, because a rubric is most definitely not a detailed model answer. (I’ve copy-pasted one of my own from last year’s ‘cellular & molecular biology’ final exam – itself adapted from an earlier Schol Bio exam – at the bottom of this post **.) The ones I use identify the key concepts/ideas that I’m looking for, plus usually a non-exclusive list of possible examples, & the mark weighting. I’ll often change them when I’m actually doing the marking, if students are writing good answers that include options I hadn’t considered (yes, it happens!). If my team’s marking term essays, then such changes are made in consultation – something that helps ensure consistency across markers. Moderation helps there, too – check-marking a couple of papers from each of the top, middle, & bottom cohorts will quickly show if another team member’s marking is consistent with mine.

And that ability to ensure consistency is important – not only so that students can be sure that their work has been marked fairly and well, but also so that if an individual’s marking is ever questioned (let’s say, for example, that a student’s not happy with their final grade & opts for a re-mark of their year’s work), then the rubrics can be made available to a new marker to use.

I should add that, when I set the term essay questions (which I really must do Very Soon Indeed), I write the rubrics at the same time & both are available to students from the beginning of the semester.You might ask, why? And I’d say, why not? Having a good rubric to hand helps the students in so many ways, in terms of learning how to structure an essay & an argument, & also in learning some of those key critical thinking skills: they need to assess the information they’re gathering & decide what’s relevant & what’s not, & how to pull it all together. The last thing I want to be reading is a series of brain dumps, where a student’s simply written everything they know in a rather incoherent manner. Nor do I have time to help each individual student who does that sort of thing – & we used to see quite a few, before I started using rubrics in this way. Providing a marking scheme in advance saves both parties time & helps the students acquire some desirable skills. (The old adage about leading horses to water still applies, alas!)

I hasten to add that the essay rubrics don’t include information on content in the way that an exam marking rubric does! I’ve added an essay example below as well ***, so you can compare the two :-)

 

**Final exam question & rubric

Mammoths are closely related genetically to African elephants and similar to them in body mass. Although mammoths became extinct around 20,000 years ago, a number of individuals have been found frozen in the Arctic permafrost. Some scientists believe that it is technically possible to clone mammoths from cells in these frozen bodies, thus ‘bringing mammoths back to life’ and producing a self-sustaining wild population.

Describe how this cloning could be done – including identifying a likely species to provide surrogate mothers – and discuss the genetic and evolutionary issues associated with such work. You could consider the impact of genetic drift, inbreeding and inbreeding depression on such a population of mammoths, and their long-term prospects for survival.

Describe how cloning could be done: Basic description of method (3 mks) Identifies African elephant as likely surrogate (1 mk) Explains reason for this choice (2 mks)	6
Genetic drift Gives definition (2) Describes impact on population gene pool (2)	4
Inbreeding Gives definition (2) Describes impact on population gene pool (2)	4
Inbreeding depression Gives definition	2
For all three of the above, Discusses impact on population’s prospects for long-term survival from a genetic perspective. Could include eg effects of decline in heterozygosity, decreased ability to respond to evolution of pathogens/parasites, decreased fecundity	4

***Term essay question & rubric

On the basis of fossil remains, Neanderthals are viewed as a sister species to Homo sapiens. Now new data from molecular biology are changing our understanding of human evolution.

Discuss the validity of the biological species concept in the light of recent molecular data from sapiens, neandertalensis, and the Denisova hominins.

 

Introduction – should include a definition of the biological species concept, and the nature of ‘sister species’.	/4
Briefly explain why Neandertals and modern humans have previously been viewed as sister species. How does this relate to the ‘out-of-Africa’ hypothesis for modern human origins?	/3 /2 /5
Outline the results of comparing neandertalensis and sapiens genomes, and the implications of these results.   What is the significance of the Denisova remains? (This should refer to the DNA analyses and their results.)	/3 /2 /5
How well does the biological species concept apply to Neandertals and modern humans, in the light of these findings? What are the implications for the ‘out-of-Africa’ hypothesis?	/6
Mark for content of essay	/20

thinking about academic reviews

In a couple of months I’m going to be involved in a review of another institution’s academic programs. So, as you might expect, the subject of reviews has been much in my mind, & it came up again yesterday when I was discussing paper content with a couple of colleagues.We were talking about a 3rd-year paper where, as it turns out, about half the class doesn’t have any formal background in a particular topic. (We will so not go into the ‘whys’ of this at the present point in time, but they have to do with alternate routes into & through a program.) This places obvious constraints on what the lecturer for this topic can actually cover, & they give a ‘review’ session at the start to try & cover the basics – really helpful for the ‘newbies’, and a quick refresher won’t do any harm to those who have encountered the material previously, either. But it also begs the question: how do we do our best to ensure that all students in that paper will have had previous exposure to some of the relevant concepts, albeit at a lower conceptual level?

And the obvious answer is, the program that this paper’s part of needs a review of its own. If a particular set of concepts are deemed important in developing a student’s understanding of the topic/subject, discipline, then we need to make sure that they’re introduced and then regularly reinforced – at progressively higher levels – as students progress through that program. And we need to look at where that information would be most apt.

As an example, let’s take part of the content I’ll be helping students to master next semester: the ideas around the Hardy-Weinberg equations. (These allow you to calculate allele and genotype frequencies in a population, given a set of assumptions about that population, & so to determine if it’s undergoing evolutionary change.) None of my first-years will have encountered this material before, so I give a broad-brush introduction & explain why the H-W equations are useful in population genetics, & that anyone intending to go on in ecology is going to encounter them again at third-year.

Which is fine, but then as a result of that inital conversation I sat down & had a think about where & when that particular set of concepts is going to be reinforced & further developed. I know that the lectures at 3rd-year are much higher-level than those I deliver, so what’s the link, the progression, between the two? Is the best place our second-year evolution paper? Probably not, as not everyone in that paper will have taken the first-year paper I’m about to teach. (So should we make it a compulsory prerequisite? I’m not convinced, as that would close off access to people with only the one biology paper but a keen interest in the history & evolution of life on earth, & I believe that would be a Bad Thing.) What ab0ut the second-year ecology paper? This is the most logical place to do that progressive build on the first-year intro, & it would segue well into the 3rd-year paper. H-W gets a mention there, but is it enough to further scaffold students into the requirements of the following year’s paper? And if the answer is ‘no’, then how do we address it – without impacting on the students’ acquisition of all the other relevant material???

I feel another review coming on…

the great class-size debate

Here in New Zealand, the compulsory education sector has recently received a lot of media & political attention (see here & here, for example), culminating in the reversal of a Ministerial decision to change pupil-teacher ratios in our primary, intermediate & secondary schools. Part of the money ‘saved’ by this move was to have gone towards improving teacher quality, a praiseworthy goal but one that so far lacks any clear mechanisms to support it (apart from a Ministry of Education statement that “[r]aising the quality of teaching will be helped by attracting higher quality applicants, raising the entry criteria for becoming a teacher and improving the quality of programmes of learning in ITE [Initial Teacher Education].”

Like most educators I know, I was concerned at the now-reversed proposal, for a number of reasons.

First up: the cuts in teacher numbers would have impacted hardest on intermediate schools with technology units – units offering technology classes both to their own students & in many cases to students from smaller ‘client’ schools. These classes give students the opportunity for a range of hands-on experiences – including science-based experiences – that they’d otherwise miss out on. At a time when primary schools have been reproached because many pupils miss out on quality learning in science, it did seem strange to put intermediate schools into a similar position by incorporating technology staffing for students in years 7 & 8 (the ‘intermediate’ years in NZ) into the curriculum staffing rations for years 2-10, with the end result that some schools stood to lose several teachers in this important learning area.

Secondly, part of the rationale for raising pupil-teacher ratios at all – and I recognise that for many schools there would probably have been little change – seems to have been the idea that class size doesn’t matter; that ‘teacher quality’ (however it’s defined) is more important. However, it’s clear from meta-analyses carried out by Prof John Hattie (then at the University of Auckland) that smaller classes do see appreciable changes in “[a]chievement, attitude, teacher morale and student satisfaction” – in classes of 10-15 students, with little effect when class sizes change from around 40 to 20. This was the case across all subjects & levels of student ability, in both primary & secondary schools. And it’s likely that one of the key factors involved in these improvements is time: the fact that in smaller classes teachers have the opportunity to spend more time with each individual student, providing feedback & reinforcement on a one-to-one basis.

For Hattie has found that

the most powerful single moderator that enhances achievement is feedback. The simplest prescription for improving education must be “dollops of feedback” — providing information how and why the child understands and misunderstands, and what directions the student must take to improve

where ‘feedback’ includes things like “reinforcement, corrective feedback, remediation and feedback, diagnosis feedback, and mastery learning” (based on that feedback). And giving that sort of feedback takes time, & quite a lot of it.

Funnily enough, just about every year when the paper & teacher appraisal results for my papers come in, my lowest score is for the statement “this teacher regularly provides me with feedback about my progress”. Now, I suppose you could say that in a class of ~200**, the opportunities for me to provide this are limited, but in fact students get feedback in class via things like pop quizzes; on Moodle – for example, through ‘common errors’ feedback almost as soon as essays are submitted; in writing, on test papers & written assignments; & face-to-face. Last year I asked the class about this – it turned out, to my surprise, that most of this was not recognised as ‘feedback’: many of them saw only verbal, face-to-face responses as feedback! This was a timely reminder that teachers and their students don’t necessarily have a common understanding around common classroom terminology.

And thirdly – well, the proposed changes did rather seem to be putting the cart before the horse, in that we seemed to be lacking a common, public, understanding on just what constitutes teacher quality, let alone how we should measure it. (For our national Tertiary Teaching Excellence Awards, the latter is done on the basis of portfolios submitted by those nominated for an award: a daunting task where there are some dozens of portfolios. I can’t imagine doing anyone the same for the 52000+ teachers in our compulsory education sector!) Despite all the heat around issues such as class sizes & performance pay, what we haven’t had is just that public discussion around what constitutes an excellent, expert teacher. There are studies (again, including work by John Hattie) that identify the attributes of such teachers. What we seem to lack is any agreement on how to apply these studies to the classroom in order to identify & esteem those experts – or any substantive discussion*** on how to encourage and support our very many other experienced teachers to join their ranks.

**The NZ Herald has covered the whole story in some depth. One of the silliest comments I’ve seen was in response to an op-ed piece by Dita di Boni, when F Max remarked that

And amazingly kids can go from a class of 30ish to a university lecture of 300+ learning far more difficult concepts. So why is the teacher ratio argument ignored at uni? Apparently our universities are in crisis and everyone must be failing. Or maybe it’s less about numbers and more about quality, something most of our teachers greatly lack.

Apart from impugning the professionalism of our classroom teachers, & ignoring the fact that the students in university classes are different in many ways from those in a primary or secondary classroom, F Max seems unaware that uni lecturers like me don’t just stand up in front of a class & lecture at them. Tutorial classes of 10-30 students give much better opportunities for feedback & one-on-one instruction – opportunities that many classroom teachers may only dream of.

*** Perhaps this is something that individual Ako Aotearoa Academy members might be interested in contributing to?