teachers’ reactions to this year’s year 13 bio exam

Today I’ve been hearing from some very unhappy teachers. As in, teachers who are upset to the point of tears on behalf of their students. Excellent, very experienced teachers. The reason for their unhappiness? This year’s NCEA Level 3 (year 13) biology exam, sat by their students just a few days ago.

And at this point I should emphasise that the teachers’ concerns were focussed towards the New Zealand Qualifications Authority, and not the individual examiner(s) who, after all, prepare these documents with advice and guidance from NZQA staff. Their concerns were focused on the system.

Now, it’s several years since I was an examiner at this level, and I know that the nature of the exam has changed. And of course the teachers themselves are well aware of what’s been expected in the past; they’re just taken aback by the nature of this year’s papers**.

Thing is, I was also involved in developing Scholarship-level exams and to me, while for some questions there’s quite a bit of resource material to get through in this year’s L3 exam, the amount of writing required of students seems an awful lot for level 3. The question-books for the exam (you’ll find them here) contain multiple blank pages for students to write their answers: the implicit message is that a lot of writing is needed. There are 3 questions like this in a book, so three essay-type answers for students hoping to achieve an excellence for the paper.

This may not sound like much – but the actual exam covers 3 separate achievement standards. So a student who’d prepared for all three (and most schools encourage this) would find themselves faced with writing up to 9 (yes, nine) extended answers over the space of three hours. (For comparison, a Schol Bio candidate would write just 3 essays in the same time frame.) In other words, the demands of this exam are such that it would quite likely preclude students from doing justice to all three papers.

So, here are some of the teachers’ concerns (I’m quoting with their permission):

• for an ‘excellence’ response, a student had to demonstrate high-order analysis & evaluation skills, in an answer generated in just 20 minutes (less, really, because of the requirement to read the question & plan an answer first). This is a big ask.
• students who were slower writers, or who had lower (but still OK) literacy levels would struggle to complete in a way that still allowed them to demonstrate their knowledge about biology and so gain an ‘achieved’. [And let’s remember that there’s a lot more to knowledge than simply being able to write a bunch of definitions.]
• “all my students felt let down by this examination. All their hard work, dedication, and love for the subject were lost in those 3 hours.”
• “I have seen a fair few of my students this week and they are so demoralised! Many are gutted they tried to do all 3 papers [because they] couldn’t do them justice- am gutted for them!”
• “A Facebook comment stated, “OK, I will play their game and only do two externals next year”. What is happening to the integrity of our subject when the assessment is driving the whole course structure of Biology in schools throughout the country?”
• “We are losing students because the subject is deemed too difficult. A colleague informed me that her daughter would not be taking Biology next year because it was too hard so she is taking Physics and Chemistry instead.”

Yes, this is anecdotal. But if these comments reflect a widespread reality, then science education in this country will be the loser.

 

** I was rather concerned about a particular question, too – but that’s best left to another post, on my ‘other’ blog.

unplugging a flipped classroom

The always-excellent Faculty Focus has been running a series on techniques for developing and running flipped classrooms. I’ve been reading them with interest, because – as some of you might remember – I’ve ‘flipped’ some (but not all) of my own teaching sessions.

Now, my own classes have been pretty low-tech; with the ‘design-an-organism’ classes (an idea that I learned from my colleague Kevin Gould), students are expected to do a bit of revision of their notes, but the actual lecture-room experience involves nothing more than group work + pens & paper (& a projector to share the results).  So the topic of a recent post naturally caught my eye: The Flipped Classroom Unplugged: three tech-free strategies for engaging students – not least because at my workplace there’s an increasing amount of discussion around ‘going digital’, and we need to take care not to throw the baby out with the bathwater.

Dr Barbi Honeycutt’s list includes: adapting the ‘muddiest point’ feedback technique (except now it’s the students who analyse the comments for commonalities and patterns); mind-mapping; and a brain-storming challenge.

I use mind-mapping quite a bit in class, & also in my own thinking & planning. Barbi’s post reminded me of the PhD research of my friend (& then-student), Cathy Buntting, in which she had me teaching students in tutorial classes how to develop mind-maps using the same tools Barbi describes: post-it notes, pens, & large sheets of paper (in place of whiteboards). (Writing concepts on the sticky notes lets students move them around, revising their maps as their understanding changes.) We also encouraged the students to use concept maps in their revision & to plan essays in exams – and Cathy found that the sort of deep learning encouraged by this technique really paid off in those examinations: students who’d learned complex information using concept maps did much better on questions testing complex understandings than those who tended to use shallow, rote learning methods. (There was no difference between the groups when it came to rote-learning tasks.) She also found that a large majority of students thoroughly enjoyed these tutorial sessions and found the mind-mapping technique both enjoyable and helpful.

I’ve used concept mapping widely (though not exclusively; a range of tools is much better) ever since. However, in lecture classes it’s usually been as a means to show how to review knowledge of a topic & plan out an exam answer, after students have spent time in discussion. In future, I really must be a lot more active in encouraging their own use of this tool in lectures, & not just in the smaller, more manageable tutorial sessions.

Thank you, Faculty Focus!

 

should we stop students using laptops during lectures?

I guess it depends on what they’re using their laptops for.

Most days when I come in at the back of the lecture room & walk down to the front, I’ll see a lot of laptops open & in use. Quite a few students will actually have the (incomplete*) powerpoint for the day’s class open on their screens, but quite a few others are on Facebook (or some arcane form of social media that I haven’t caught up with yet) or just surfing. So when a friend shared an article titled Professors push back against laptops in the lecture hall, I read it with interest & also shared it with one of our big FB student pages for some consumer opinion. (There’s some interesting commentary here, too.)

One of the major reasons many oppose laptop use is their potential to distract students from what’s going on in the classroom, and judging from the ‘consumer feedback’ I received, that can be quite a big issue:

I don’t begrudge others using them except when they are watching videos or checking facebook etc during lectures. That’s very distracting.

It’s only annoying and distracting when people take their laptops and play games or scroll Facebook. Which a lot of people do…

Somewhat surprisingly, that distractive effect extends to students putting their devices to what many of us would regard as ‘legitimate’ use ie searching for information directly related to the class. And I’ll admit, sometimes I’ll ask a student to look something up, especially if I think they’re doing something other than class-related work! For example, this brief report cites a study showing that

students who spent a greater proportion of time seeking course-related sites recalled significantly less than those who were more often browsing sites unrelated to the course (r = -.516, p. < .02).

And worse:

the more students used their laptops, the lower their class performance (β = -.179, t(115) = -2.286, p = .024), the less attention they paid to lectures (p = .049), the less clear lectures seemed to them (p = .049), and the less they felt they understood the course material (p = .024)

Yikes! This really piqued my interest, & led me to a 2014 paper by Mueller & Oppenheimer, which has the wonderful title, The Pen is Mightier than the Keyboard. Here’s the abstract:

Taking notes on laptops rather than in longhand is increasingly common. Many researchers have suggested that laptop note taking is less effective than longhand note taking for learning. Prior studies have primarily focused on students’ capacity for multitasking and distraction when using laptops. The present research suggests that even when laptops are used solely to take notes, they may still be impairing learning because their use results in shallower processing. In three studies, we found that students who took notes on laptops performed worse on conceptual questions than students who took notes longhand. We show that whereas taking more notes can be beneficial, laptop note takers’ tendency to transcribe lectures verbatim rather than processing information and reframing it in their own words is detrimental to learning.

I’ve certainly observed that many students struggle with long-hand note-taking, to the extent that I’ll get the occasional complaint that “she moves on to the next slide before I’ve copied it all down” in my teaching appraisals. (I do explain that they shouldn’t be ‘copying it all down’…**) And I type much faster than I write, so I can sympathise with students who want to use their laptops for note-taking in class. So did some of my students, commenting that

I actually find typing notes better for me, because my typing speed is so much faster than my writing speed.

and

I would hate it if we were not allowed laptops in lectures anymore! I’d miss half the notes and then have to go home and panopto lectures (or die if they weren’t panoptoed) which just takes up time that i could use studying all my notes properly.

Mueller & Oppenheimer’s paper has really got me thinking. They point out that there is a considerable body of evidence around the efficacy of note-taking, commenting that even without the distraction effect,

laptop use might impair performance by affecting the manner and quality of in-class note taking.

This could have that negative impact on learning by two routes: ‘encoding’, and ‘external storage’. ‘Encoding’ is valuable because – ideally! – students process information as they make their notes, and doing this enhances both their learning & their ability to retain information. ‘External storage’ refers to the ability to review and learn from notes at some later point, including notes taken by others: indeed, we employ note-takers to do this for students who are unable (for a variety of reasons) to take notes themselves.

An important question here is, what are students actually doing when they take those in-class notes? Are they actively summarising what’s been discussed eg via drawing a concept map, or writing a paraphrase? Or are they simply copying, word for word, every single thing I say & show in class?*** While some could argue, “but it doesn’t matter ‘cos I’ll write a summary later”, Mueller & Oppenheimer observe that

verbatim note taking predicts poorer performance than nonverbatim note taking, especially on integrative and conceptual items.

This underlies their suggestion that while laptops allow more rapid note-taking, if those notes are verbatim, then learning and understanding may actually suffer. In fact, they observe that

One might think that the detriments to encoding would be partially offset by the fact that verbatim transcription would leave a more complete record for external storage, which would allow for better studying from those notes. However, we found the opposite—even when allowed to review notes after a week’s delay, participants who had taken notes with laptops performed worse on tests of both factual content and conceptual understanding, relative to participants who had taken notes longhand.

So where do we go from here? I must admit to being a tad flummoxed at the moment – with the need to offer more flexible learning opportunities and  the current trend to ‘paperless offices’, we’re moving into a more highly digitised world and those laptops aren’t going to go away any time soon. How, then, to overcome the apparent negative effects they may have on student learning? If part of the problem lies with the ability to take appropriate notes, do we need to somehow teach this skill to all our incoming first-years?

 

* I mean, why would I give them the whole lot up front (including the answers to my in-class quizzes)?

** no, seriously! What I’d much prefer is that they read through the material I provide ahead of class, identify the bits where they have no idea what I’m talking about, & then that’s where they should focus any note-taking during class.

*** and if they are taking such fulsome notes – how much attention is being paid to everything else that’s going on in class: the questions, discussion, extra explanations?

 

P.A.Mueller & D.M.Oppenheimer (2014) The Pen is Mightier than the Keyboard. Psychological Science 25(6): 1159-1168 doi: 10.1177/0956797614524581

a surprising misconception

I spent much of the weekend marking first-year biology exam papers. It was a lovely weekend & I really didn’t want to miss all the nice weather, so I ended up finishing the task well after midnight last night. And in the process I identified evidence of what is, on the surface, a really puzzling misconception, one that relates to the effects of X-chromosome inactivation.

Now, we’d spent quite a while in class discussing X-chromosome inactivation in female mammals: why it happens, how it happens, & its phenotypic effects (anhydrotic ectodermal dysplasia, anyone?). One of the images I used in this discussion was of Venus, a tortoiseshell cat with an extremely unusual colour pattern:

This image comes from the NBC News site, but Venus is a very famous purrball who even has her own Facebook page, and I’ve blogged about her previously. She’s either a chimera, or we’re seeing a most unusual (but not unique) example of the typical X-inactivation tortoiseshell coat pattern. Anyway, I used a similar image of Venus and asked

What is the most likely explanation for the colour pattern shown in the coat of this female cat?

And about 90% of the class answered, “co-dominance”. Which really made me stop & think.

Why? Because it suggests that, while I’m sure they could quote me chapter and verse regarding a definition of co-dominance, they haven’t really thought any further about what that means in phenotypic terms. For if codominance were in play here, with both alleles for coat colour being expressed in each cell where the gene’s active, then we shouldn’t see that clear definition of the two halves of the cat’s face. Instead, both should be a fleckled mix (is ‘fleckled’ a word? Yes, it is; Shakespeare for the win once more) of black & golden hairs (rather like roan coats in cattle & horses).

And this gives me pause – & cause – for thought, because this isn’t a mix-up that I’d have even considered before. Is ‘codominance’ their shorthand for one gene, or the other, being expressed (due to X-inactivation)? Or do they really think that’s how codominance works? If so, it does suggest that a) I didn’t really explain codominance (or X-inactivation) all that well this year, & b) I need to review what I do before teaching that particular session again.

 

 

“If you’re going to get lectured at, you might as well be at home in bunny slippers”

This is a post I first wrote for the Bioblog.

There’s an increasing body of literature demonstrating the benefits of active learning for tertiary students taking science subjects. This is a topic I’ve written about before, but I’m always interested in reading more on the subject. And let’s face it, the more evidence the better, when you’re wanting to get lecturers in the sciences engaged in discussion around different ways of teaching. As you’ll have gathered, I find a lot of new science & education material via alerts on Facebook, as well as through the more conventional journal feeds & email alerts, and so it was with this recent paper by Scott Freeman & colleagues, which looks at the effect of active learning on student performance in science, technology, engineering and maths (STEM) classes: I saw it first described in this post¹ (whence also comes the quote I’ve used as my title).

The paper by Freeman et al (2014) is a meta-analysis of more than 200 studies of teaching methods used in STEM classes, which included “occasional group problem-solving, worksheets or tutorials completed during class, use of personal response systems with or without peer instruction, and studio or workshop course designs” (ibid.). The impact of the various methods on student learning was measured in two ways: by comparing scores on the same or similar examinations or concept inventories; and by looking at the percentage of students who failed a course.

What did their results show? FIrstly, that students’ mean scores in exams assessing work covered in active learning classes improved by around 6% over more traditional teaching-&-learning formats (& finding that matches those of earlier studies); and secondly, that students in those traditional classes “were 1.5 times more likely to fail”, compared to students given in-class opportunities for active learning (with a ‘raw failure’ rate averaging 33.8% in traditional lecturing classes and 21.8% in more active classes). These results held across all STEM subjects. The researchers also found that active-learning techniques had a stronger effect on concept inventories compared to formal exams (& here I’m wondering if that doesn’t reflect – at least in part – the nature of the exams themselves eg did they give opportunities to demonstrate deep learning?) Interestingly, while the positive impact of active learning was seen across all class sizes, it was more pronounced in classes of less than 50 students.

On the class size thing, I’m wondering if that might be because it’s easier to get everyone actively involved, in a smaller class? For example, I’ve got a colleague at another institution who runs a lot of his classes as ‘flipped’ sessions, and ensures that all students get the opportunity to present to the rest of the group – this is far easier to set up in a class of 50 than in a group with 200+ students in it. (I know! When I run ‘design-a-plant/animal’ sessions, there’s time for only a sub-set of student ‘teams’ to present their creatures to the rest of the class. Plus you really have to work at making sure you get around all teams to talk with them, answer questions, & so on, and so it’s perhaps more likely that someone can remain uninvolved.)

The research team concluded:

Finally, the data suggest that STEM instructors may begin to question the continued use of traditional lecturing in everyday practice, especially in light of recent work indicating that active learning confers disproportionate benefits for STEM students from disadvantaged backgrounds and for female students in male-dominated fields. Although traditional lecturing has dominated undergraduate instruction for most of a millenium and continues to have strong advocates, current evidence suggests that a constructivist “ask, don’t tell” approach may lead to strong increases in student performance, amplifying recent calls from policy-makers and researchers to support faculty who are transforming their STEM courses.

The ‘bunny slippers’ quote from the lead author comes from the post that originally caught my eye. And I suspect there may well be bunny slippers (or the equivalent) in evidence when my own students watch lecture recordings at home :) But this does raise a question around massive open on-line courses (MOOCs), which tend to have a very high ‘fail’ rate – how much of this might be attributed to the difficulty in ensuring opportunities for active learning in these ‘distance’ classes?

And of course, we aren’t really talking a simple dichotomy between ‘traditional’ lecture classes and classes with a very high component of active-learning opportunities – something the research team also note: some of the ‘non-traditional’ methods they surveyed had only a 10-15% ‘active’ component. This is something discussed at more length by Alex Smith in a post entitled “In Defence of the Lecture”. I have to say that his approach sounds very similar to mine, with its mix of socratic questioning, pop quizzes, group discussions, and – yes – sections of ‘lecture’. As Small says:

Not every lecture is a person spending an hour talking nonstop to deliver facts. A good lecture is engaging, it naturally invites discussion and dialogue, it operates at a level much higher than raw information delivery, it is a natural setting for the expert to act as a role model, and it can be integrated with more formal activities (e.g., clicker questions, small-group discussions, etc.).

Lecture should not be the sole means of instruction, and bad lectures are a plague demanding eradication, but we err when we too strenuously inveigh against the lecture.

I couldn’t agree more. And maybe that’s a message that’s being lost in the louder discussion around active learning, and which needs to be heard more widely.

¹ The comments thread for this story is also worth reading.

S.Freeman, S.L.Eddy, M.McDonough, M.K.Smith,N.Okorofor, H.Jordt & M.P.Wenderoth  (2014) Active learning increases student performance in science, engineering, and mathematics. http://www.pnas.org/content/early/2014/05/08/1319030111