thoughts on the proposed changes to NCEA

This post was first published on the Bioblog.

Many readers will probably have read this RNZ article (or heard the related interview), or seen calls for consultation on the Ministry of Education’s suggested changes to the number of subjects – and achievement standards – on offer to year 11 students.

I’ve been following (& participating, where I can) all this with colleagues and friends, and thought I’d share some of my thoughts here. But before I get onto that, I’ll point out that there’s been a fair bit of consultation even before we got to the point where these materials have gone out, in their turn, for feedback. That process began in 2018 and resulted in a “change package“. This was published in May 2019, and I really recommend reading it carefully as it provides the rationale for the latest 2 rounds of consultation (about the draft L1 Science standards & their supporting material, and about the number of individual subjects that should be offered to year 11 students.

In the interests of full disclosure, I’m a member of the Subject Expert Group (SEG) that is working on the draft L1 Science achievement standards.

So, the SEG members were tasked by the Ministry with developing four Science achievement standards (ASs), but that decision on the number of standards was based on a lot of feedback from a wide range of sector & interest groups, which signalled very clearly a need to reduce the complexity of NCEA & reduce the number of standards¹.

I’ll admit that one of my concerns regarding these two recent consultation rounds is the overlap between requests for feedback about the initial drafts of the Science material, and the announcement of consultation on the number of subjects on offer. I think it’s meant that people have conflated the two.

But – none of this is set in stone; it’s all draft material. Feel strongly about it? Then follow the appropriate links above, and be heard. And – read all the relevant materials before you comment.

One of the things I’ve heard quite often about the Science ASs is that the actual subject material is “hidden”. To some degree this might be due to people reading the headlines, and the ASs, and not also going through the supporting material: the learning matrix (which clearly identifies content) or the Teaching, Learning & Assessment Guide (TLAG for short). But from my perspective, the content material for biology, physics, chemistry, and earth & space science remains the same, and provides an essential context for delivering concepts and competencies relating to the Nature of Science strand in the National Curriculum document (NZC). Hopefully the next round of consultation documents will see the inclusion of some examples of teaching and assessment plans that show what this would look like in practice.

Thus, I think there does need to be an element of trust that teachers will continue to deliver content, & in fact – speaking personally – I would hope there will be a clear statement at some point about the need to cover content. However, I also think it’s important to remember that at the moment there are 31 standards available to schools delivering a year 11 Science program (which is almost all of them) and thus there is no guarantee of consistency now about what content students may or may not have covered.

I’ve heard a lot of concern about the need for professional learning development (PLD) opportunities for teachers. It’s a concern that I know is shared by all of us on the SEG, and it’s one that we’ve communicated to the Ministry. This is a shift in direction; it will entail a significant amount of work by classroom teachers; and there absolutely needs to be a substantial amount of PLD available well before implementation of any confirmed changes to the NCEA. (Not least, for science teachers, because the year 11 changes will probably flow down – to year 9 & 10 classrooms – and may have some impact ‘upwards’ as well.

But – & it’s a very big ‘but’ – I think that it would be easy to lose sight of the fact that the proposed standards are very much aligned to the NZC in placing  the nature of science front & centre (its delivery to date, if present, has been largely implicit).  As I wrote in my previous post,

Back in 2007 New Zealand implemented a new national curriculum. One of the features of the science component of that document is the overarching importance of students gaining an understanding of the nature of science (the “unifying strand” of the curriculum). In that context, it expects that:

students learn what science is and how scientists work. They develop the skills, attitudes, and values to build a foundation for understanding the world. They come to appreciate that while scientific knowledge is durable, it is also constantly re-evaluated in the light of new evidence. They learn how scientists carry out investigations, and they come to see science as a socially valuable knowledge system. They learn how science ideas are communicated and to make links between scientific knowledge and everyday decisions and actions.

And the document specifically adds that these outcomes are pursued through the following major contexts (the various science ‘subjects’) in which scientific knowledge has developed and continues to develop.

 

Given that currently about 60% of students in year 11 science don’t go on to further study in any of the sciences, I’d argue that while a scientifically-literate society does need some knowledge of science, it also requires a solid understanding of the nature of science itself.

 

 

¹ In my personal opinion, the inclusion of additional specific subject standards at year 11 would pretty much destroy the kaupapa of the SEG’s work, in that we would not see students gaining that key, core understanding of NoS. The nature of the 4 ASs currently out there for feedback was not determined randomly, but as the result of a fair bit of thought and discussion by the SEG members.

why do students need to learn about the nature of science?

This post was first published on the Bioblog.

You’re probably aware that the Achievement Standards used to assess senior school students’ learning are being reviewed. Science is one of the ‘pilot’ subjects in this process, where a ‘Subject Expert Group’ has developed 4 draft Science standards¹ (a significant step away from the current 30+, and a response to advice from several high-level advisory groups). These drafts have been out for consultation, and are all intended to develop and assess students’ understanding of the nature of science, with subject content providing the contexts for this learning. (That is, the subject content has definitely not disappeared.)

Why is this important?

Back in 2007 New Zealand implemented a new national curriculum. One of the features of the science component of that document is the overarching importance of students gaining an understanding of the nature of science (the “unifying strand” of the curriculum). In that context, it expects that:

students learn what science is and how scientists work. They develop the skills, attitudes, and values to build a foundation for understanding the world. They come to appreciate that while scientific knowledge is durable, it is also constantly re-evaluated in the light of new evidence. They learn how scientists carry out investigations, and they come to see science as a socially valuable knowledge system. They learn how science ideas are communicated and to make links between scientific knowledge and everyday decisions and actions.

And the document specifically adds that these outcomes are pursued through the following major contexts in which scientific knowledge has developed and continues to develop.

The development of that list recognised that the country’s future prosperity depends on students continuing to study science and entering science-related careers. This is because – as the late Sir Paul Callaghan observed –‘rich’ countries depend on high-end science and technology, and NZ needs to invest far more heavily in these fields to maintain and enhance its standard of living. That is, we need more scientists, scientifically-literate politicians, and a community that understands what science is done and why it’s relevant to everyday life.

But in practice, since then we’ve probably focused more on subject content than on explicitly teaching what science is, how it works, why it is such a powerful tool for understanding the world around it, and that it is a human/social endeavour. (I’m sure it’s implicit in many programs, but things like this aren’t universally picked up by osmosis: practice reinforces learning.)

Does this matter?

Well, yes it does. Knowledge of content is important, but I’d argue that it is far from being enough. Around 60% of year 11 (NCEA L1) students won’t go on to take science subjects at year 12 or 13. They need – all students need – more than content to be science-literate (as this recent PISA document makes clear). To that end, the NZ Curriculum document asked that in addition to content knowledge, students gain the ability to critically evaluate science ideas and processes; to communicate about science; and to recognise that science is a human endeavour² (people develop our scientific knowledge and that their ideas change over time).

And having the knowledge, understandings, and competencies that should be delivered by a teaching & learning program assessed using these standards, students should then be able to critically engage with the various science-based & science-informed issues that they’ll encounter, now & in the future. (And to deal with claims such as “well, science got it wrong in the past, so it can’t be trusted now”; and “science is always changing its mind”, both of which are hallmarks of those arguing against established scientific knowledge.)

That’s what the draft standards are intended to deliver, together with the acquisition of content knowledge. And I think that’s a very good thing.

 

¹ disclosure: I am a member of this group.

² The concept that science is a human endeavour is explicit in the title of one of the draft standards.

the sad state of science learning in primary school

This post was first published on my ‘other’ blog. It’s not intended to diss primary school teachers – quite the reverse! They need all the help & support they can get to help them deliver the science curriculum.

In 2011, Sir Peter Gluckman released his report, Looking ahead: science education for the 21st century. In it, he noted the need to improve science teaching in primary schools, commenting that

there should be an attempt to improve the confidence [my emphasis] of all teachers within primary schools to assist in science and that all primary schools should be encouraged to develop a science champion.

And in 2012, David Vannier pointed out that

there is growing evidence that too many children are not doing well in science and do not have access to effective instruction, especially at the primary level.

and that

[at] the same time that the New Zealand government is seeking to spur innovation in science as a means to improve the economy, less and less emphasis is being placed on science instruction in primary schools.

Fast forward to Monday this week, when Radio NZ reported on the findings of The National Monitoring Study of Student Assessment (NSSA): that 20 percent of Year 8 children last year reached the expected level of achievement in science – the lowest figure of any learning area in the curriculum. While most children liked learning about science at school – 82% of those in year 4 and 65% in year 8 – those figures haven’t changed significantly since the previous survey in 2010, and the decline between years 4 and 8 should be a concern. Overall, these results don’t augur well for science literacy and engagement with science amongst our young people.

You may be tempted to lay this result at the feet of National Standards. Don’t. Looking Ahead was published in 2011. National Standards were first implemented in 2010, just a year earlier. The issues identified by Sir Peter Gluckman have had a longer gestation than that.

I wrote about Sir Peter’s report at the time, highlighting his statement that

science education is not just for those who see their careers involving science but is an essential component of core knowledge that every member of our society requires.

Thus, science education needs to deliver on what Sir Peter characterised as ‘citizen-focused objectives’, where all children need to have:

• a practical knowledge at some level of how things work;
• some knowledge of how the scientific process operates and some level of scientific literacy
• enough knowledge of scientific thinking as part of their development of general intellectual skills so that they are able to distinguish reliable information from less reliable information.

But can it deliver? His report also notes that

[a] well prepared primary school teacher will integrate excitement about the natural world and scientific forms of thinking into literacy and numeracy teaching, and into general educational processes. The challenge is how to provide primary teachers with the skills to do so. [My emphasis]

I believe that meeting this challenge will require changes to at least two things: teacher-training curricula, and professional development (PD) and support.

Just 25% of primary school teachers hold another qualification, in addition to their teaching degree, and it’s probably fair to say that BSc graduates are in a minority. Intending primary school teachers usually study for a 3-year Bachelor of Teaching degree, and take a range of papers in their first year – including one on science teaching. This one paper, plus learning opportunities while on practicum in schools, may well be their sole exposure to science (Campbell, 2018).

Which is where the PD and support come in. Ally Bull (2016) found that science was “marginalised” in the primary curriculum; and that teachers – lacking confidence to teach the subject – often had little in-school support and only limited access to opportunities for PD. The majority of those providing the PD (51%) aimed to enhance teachers’ confidence to teach science, and just 5% felt that developing their knowledge of science was important. Bull also cited other research that found that “primary teachers’ lower confidence in low confidence in teaching science reflected their lesser degree of content knowledge.”

There are ways to address this. Anne Hume & Cathy Buntting (2014) developed resources and shared these with primary teacher trainees, encouraging them to think about what science ideas they could teach (plus the why, when & how) while using those resources. Their results? Really encouraging:

Even student teachers who had previously felt very apprehensive about teaching science reported feeling far more confident about the prospect after completing the CoRe assignment.

Programs like theirs, changes in teacher education, and the commitment to provide ongoing mentoring and support, should raise teachers’ confidence in teaching science and see them reach their full potential as ‘science champions’. Our teachers and our children deserve no less.

 

 

talking about what we should teach

This is a cross-post of something I originally wrote for my ‘other’ blog.

While I was on holiday (Japan – it was wonderful!) – I read Tom Haig’s interesting article about ‘curriculum wars’ over on Education Central, and it reminded me of the concerns I’ve held for some time that we don’t really talk enough about what to teach in our classrooms, be they university-level or in the secondary sector.

Several years back (how time flies!) I was involved in developing the ‘Living World’ component of the New Zealand Curriculum document, as well as entering into the discussions around what the science component of that document should deliver. (Right down to a discussion of what it actaully is to ‘do’ science.) At the time I was somewhat taken aback to discover that the panel was not required to give any exemplars for teachers, any indication of what they might do to help students master particular concepts – something that’s noted by Tom. Yes, I totally get it that schools are free to set their own curricula, but at the same time I couldn’t halp thinking that the occasional ‘starter for 10’ might be useful.

Layered on top of that – & amplified by my experiences in relation to developing and assessing Achievement Standards for NCEA, was the way that while new content or concepts might be loaded on up-front, we didn’t seem to remove stuff at the other end. This had the result that the amount of information associated with a standard might just grown & grow (CRISPR, anyone?). Pretty much the same thing tends to happen at university – if you look at one of the standard first-year biology textbooks, Campbell Biology^A, you’ll see that it’s become steadily thicker over time as new material’s added. (In my experience, at least some first-year uni lecturers argue that all the basic stuff should be delivered at school; they shouldn’t have to teach that. However, this sits poorly against the fact that no NZ universities have any prerequisites for their first-year biology papers, and also suggests that those making the statement don’t really recognise that not all year 13 students are heading for university. Remember, schools have the ability to shape their curricula to suit the needs and requirements of their individual communities.)

In other words, we didn’t seem to be having any discussion around what should be taught, and why. And we still don’t, although hopefully such issues will be addressed in the review of NCEA. For, as Tom Haig says:

Working out what we should be teaching, and why, is something that we should be discussing together and taking much more seriously as teachers than the second place it’s taken to discussions of technique. Hattie, ERO, the Best Evidence Synthesis and so forth are filled with advice about ‘how’, but shouldn’t we be thinking just as hard about ‘what’?

^A No relation! I was privileged, though, to meet the late Neil Campbell when he visited New Zealand, and was struck by what a wonderful educator he was.

what are the challenges for first-year core science courses?

This is another post based on a talk at FYSEC2017, & which I’ve also published on my bioblog.

Prof Karen Burke da Silva was the keynote speaker at Day 1 of the 2017 First-Year Science Educators’ Colloquium, held in Wellington. Her topic:Transforming large first year science classes: A comprehensive approach to student engagement. Currently at Flinders University, she’s been instrumental in setting up an ‘integrated teaching environment’ that’s seen a drop in withdrawals, and a marked increase in engagement, among their first-year STEM students.

If you’ve read my earlier FYSEC-focused post, you’ll know that student engagement was a hot topic at last year’s colloquium. Which isn’t surprising; as Karen noted, both NZ and Australian universities have trouble with attention, engagement, retention, and performance of their first-years, who face some significant challenges in transitioning from their smaller high-school classes to the large lecture rooms of universities. She commented that

how best to build a first-year program in sciences that allows for different student backgrounds, abilities and interests is a task that all first-year coordinators face.

Because students are so diverse, if we’re going to accommodate their various needs and backgrounds, we really need to know about those first. In Australia, the SSEE Project gathered data on both student and staff expectations and experiences (& whether the two converged) across all disciplines at Flinders, the University of Adelaide, and the University of South Australia. The decision to set this research project up was based on some reasonably concerning information:

• Statistics show that of all students entering Australian universities one-third fail to graduate and of those students who withdraw from their programs over half withdraw in their first year.
• Students preparing for tertiary study may do so individually or via school, government and university initiatives. Many students, however, still experience an early ‘reality shock’ during their first semester rather than a smooth transition to university.
• The mismatch between students’ expectations and experiences has ramifications for their learning, satisfaction, retention and ultimately, their wellbeing.

Among the findings that Karen presented to us:

the majority of students were neutral about, or agreed with, the statement that secondary school education was an adequate preparation for university study;

students from schools offering the International Baccalaureate program outperformed those from all other schools on entering university (with students from state schools doing least well), & the difference was still reflected in GPAs at the end of that first year:  However, the difference between state and private schools disappeared over that time;

friends, university websites, and universities’ recruiting efforts had more effect on shaping students’ views about university study than teachers, guidance counsellors, family, new and traditional media outlets, and provided a more accurate reflection of what uni life is really like.

students’ expectations around what constituted a reasonable time interval for returning marked work to them were not matched by the reality: the majority expected it back in 2-3 weeks, but in reality most waited 3-4 weeks;

the great majority felt that receiving feedback on drafts would be very important to their learning – but most disagreed with, or were neutral about, the statement that they actually received such feedback. (While students may not be aware that there’s more to feedback than written comments on an assignment, providing feedback in a timely manner is something that most universities need to work on.)

When Karen arrived at Flinders, back in 2007, the STEM disciplines had a high fail rate of around 23%; this was particularly noticeable among mature students & those who hadn’t taken the final year of high school. The changes she & her team made to teaching delivery were intended to address this, but they would have the effect of enhancing the learning experience for every student. I found her ideas around this really exciting (although I suspect that those wedded to a more ‘traditional’ approach to delivery would be shaking their heads).

This is what the new program looked like: first up, the first semester of the year became ‘transitional’, ensuring that everyone was in the same place before entering semester 2, which was ‘extension’, taking students’ knowledge & understanding further. Along that were ‘pre-lecture’ classes^A for students identified as lacking the normally-expected background in the subject, which resulted in the students having greater confidence in their ability to cope with the subject, plus increased motivation & understanding. And I loved  the idea of regular case-based ‘lectorials’, where the students were actively engaged in addressing the issues raised in each case study. Karen’s research showed that 98% of students reported that these classes enhanced their understanding of how biology relates to the real world.

Learning was further supported by peer-assisted study sessions, run by 2nd- & 3rd-year students (who received training for the role), which were part of the formal timetable and for which students could gain up to 5% of their final grade for attendance. Karen reported that these sessions were very well attended.

And of course, STEM subjects have labs. Karen told us that Australian universities are tending to reduce the lab component of STEM papers, such that most first-year papers have less than 30 hours of practical classes – this is a real pity as in general students really enjoy labs and the practical classes (if properly focused) can enhance understanding of key concepts as well as teaching a range of practical skills. (I’m often perplexed by suggestions that we move to on-line ‘labs’, as both lab & field work have a lot of practical & interpersonal skills development associated with them, & that’s something that you don’t get by interacting with a mouse & a screen.) At Flinders, Karen told us that a science paper would have 2, two-hour, lab classes each fortnight: the first session is all about preparation & planning, & the second is the actual practical work. It seems to me that this would give students a good experience of actually ‘doing’ science – something that the students agreed with, as well as reporting that they liked becoming more responsible for their own learning.

The research projects that all science students at Flinders do in their first year of study would also have that effect, although they have to be scaffolded into these assignments – which also provide an excellent opportunity to learn many of the personal skills needed for successful teamwork. (This is another of those competencies that universities often say their students gain, but for which they often don’t really provide much in the way of carefully-designed learning opportunities.)

I was fascinated to hear that Karen also includes art, & other creative tasks, in her assessment tools – this is great as it allows students to recognise that science contains an element of creativity. She commented that having the first assignment as an art project both helps to remove the fear associated with doing a science assignment, and helps connect the teacher with their students. The question she sets is a very simple one: what does biology mean to you? These were self-graded, something that would make many science lecturers raise their eyebrows! – but apparently in moderating the results Karen’s found that 90% of the class awarded themselves the same marks that she would. Of the remaining 10%, those who graded themselves lower tended to be female, while those giving a higher mark were male. The students submitted some amazing work.

Apparently other staff weren’t always happy as they felt that students didn’t give their own assignments the same attention – but there was a happy outcome: they began to look at ways of offering the opportunity for similar assignments, with a real-world focus, in their own papers. I’d do that myself, given that these changes in delivery & assessment had a marked impact in terms of student outcomes, with fewer failures & withdrawals.

And we were reminded that students need to feel some connection with the institution & with those teaching them. (There’s quite a lot of literature available on this, including TLRI studies from NZ & other papers like this.) Having that contact offers opportunities to find out how the paper is progressing, & also to identify any problems that students might be having & to refer the students to appropriate support if necessary. I think it would also help lecturers to understand the school system that our students have come from; having that understanding is crucial in optimising the transition from secondary to tertiary learning environments.

We ended with some questions around the value of recording lectures. My institution does this; I suspect most universities in NZ do. Feedback from students indicates that the practice is helpful for international students, those wanting to review their understanding, & for those who’ve had to miss a class; Ican certainly see the peak in views just before a test! But we’re finding that many students neither attend class, nor view the recordings, & while some may muddle through like this, others don’t. So, we need to come up with a way to change students’ mindsets – and for their seemingly insatiable demand for recordings & lecture notes & previous exams. (This is something that’s definitely a carry-over from school, I think.) So, how do we deal with that demand, that sense of entitlement, that lack of engagement? I’m not sure I have the answers. Do you?

Karen thinks recorded lectures have changed face of education in a very negative way. Good for internationals, for high-achievers, for review. But the mid-range group don’t show, don’t view the recordings either. If we’re to continue with recordings then we need to change the student mindset as well.

^A For those interested in the concept of prelectures, here’s the abstract from one of Karen’s papers on the subject:

First year biology students at Flinders University with no prior biology background knowledge fail at almost twice the rate as those with a background. To remedy this discrepancy we enabled students to attend a weekly series of pre-lectures aimed at providing basic biological concepts, thereby removing the need for students to complete a prerequisite course. The overall failure rate of first year biology students was lowered and the gap between students with and without the background knowledge was significantly reduced. The overall effect of the implementation of pre-lectures was a more appropriate level of teaching for the first year students, neither too difficult for students without a prior biology background and no longer too easy (or repetitive) for students with high school level biology.

more on laptops in lectures

I type much more quickly than I write (some would argue, also more legibly). But when I’m taking notes in meetings, I do it with a (very old-fashioned) fountain pen & notebook. The reason is that this makes me filter what I’m writing, so that only the relevant points make it onto paper.  And this is why I’m actually somewhat chary of requiring, or expecting, students to take lecture notes on laptops, despite the push in many quarters for ‘bring your own device’ (BYOD) to classes in the expectation that students will do just that.

Yes, there are some good things about using laptops in class (see here, for example – it’s a commercial site but I ignored the little pop-ups wanting to sell me things). They allow for faster note-taking, & if students are using google docs for that, then they can access their notes anywhere – they can also collaborate on the notes, which offers some exciting possibilities for peer-assisted learning. Laptops & other devices can also increase engagement eg via using them to complete in-class quizzes & polls.

However, they also allow for people to feel that they are multi-tasking – tweeting (as many academics do at conferences these days), chatting on messenger, posting on Facebook. Unfortunately that means that their attention’s divided and their focus on learning is diminished. It could be – and has been – argued that that’s the educator’s fault; that we should offer such engaging classes that no-one’s interested in goofing off, and indeed I think there is some truth in that. After all, if what the lecturer says is pretty much identical to what’s in the slides they posted on line, many students may not see much incentive to pay attention, because “I can always read the notes or watch the recordings later”. (Only, many never do :( )

What’s more, the off-task use can be distracting to other students as well as the individual users:

We found that participants who multitasked on a laptop during a lecture scored lower on a test compared to those who did not multitask, and participants who were in direct view of a multitasking peer scored lower on a test compared to those who were not. The results demonstrate that multitasking on a laptop poses a significant distraction to both users and fellow students and can be detrimental to comprehension of lecture content (Sana, Weston & Cepeda, 2012)

and

Most importantly, the level of laptop use was negatively related to several measures of student learning, including self-reported understanding of course material and overall course performance (Fried, 2006)

and

Results show a significant negative correlation between in-class phone use and final grades… These findings are consistent with research (Ophir, Nass, and Wagner 2009) suggesting students cannot multitask nearly as effectively as they think they can (Duncan, Hoekstra & Wilcox, 2012).

Laptops & tablets also allow for very rapid note-taking – and yes, I’m saying that like it’s a bad thing. But if you’re typing so quickly that you can take down what’s being said verbatim, then you’re probably not processing the information, and that has a negative effect on learning and mastery of the material further down the track. This was investigated by Mueller & Oppenheimer (2014), who found that even when students were completely on task i.e. using their devices only for note-taking, their engagement and understanding was poorer than those taking notes longhand. (That’s in addition to other negative impacts they identify: students off-task, poorer academic performance, and even being “actually less satisfied with their education than their peers who do not use laptops in class.”)

Mueller & Oppenheimer cite earlier work that identified two possible, positive, impacts of longhand note-taking: the material is processed as the notes are made, which improves both learning (makes it more likely that deep, rather than shallow, learning will occur) and retention of concepts; and the information can be reviewed later (of course, that’s also possible with digital notes).  Processing usually involves paraphrases &/or summaries – which is what my meeting notes generally look like – but can also involve tools such as concept mapping, and there’s a lot of research showing that students involved in this sort of activity do better on tests of conceptual understanding and the ability to integrate information.

So, since it’s those higher-order skills that we hope to develop in our students, perhaps we need to tread carefully around the BYOD idea. Or at the very least, discuss all these issues with students at the start of the semester!

 

 

helping first-year students cope with the reality of university study

For many students making the transition from secondary school to university can be a difficult experience. Their teachers have probably told them that they can be expected to learn more & work harder, but the students don’t really know what that entails beyond doing ‘more of the same’. (They may also have been told that at uni it’s ‘sink or swim’, & that they’ll be left pretty much to their own devices – it was nice to hear from a group of our class reps that they hadn’t found this to be the case and that they felt their learning was well supported.)

Unfortunately doing more of the same, and just doing it harder, may not be a good coping strategy when it comes to self-directed learning. Certainly our experience in first-year biology this year was that many students simply seemed unaware of, or unprepared for, the need to do more than simply attend lectures (or watch them on panopto).  And as Maryellen Weimer points out in her excellent blog on The Teaching Professor, there are an awful lot of distractions: new friends, new social opportunities, new jobs… Plus students can find it hard to recognise when they do or don’t understand something, equating familiarity with knowledge, and are used to a lot more teacher guidance.

And as Maryellen points out,

Additionally, there’s the reluctance of students to change their approaches. When asked what they plan to do differently for the next exam, students often respond that they’ll do what they did for the previous one, only they’ll do it more. Dembo and Seli’s research shows that even after successfully completing developmental courses that teach learning strategies, students didn’t change their approaches. Finally, and even more fundamentally, strategies may be known and understood, but unless they’re applied, they’re worthless.

This is something I hear quite often, from students who’ve been asked to see me because their teachers have identified that the students are struggling. The idea that continuing to do the same, but more & harder, is a hard one to shift sometimes.

But over on The Teaching Professor, you’ll find some useful suggestions for turning this around. Some of these, such as moving to earlier assessment, are changes we’re already making, but there are clearly other tools to use as well. And as our student cohorts’ demographics continue to change (we’re seeing an increasing number of first-in-family enrolments, for example), there’s an urgent need for universities to adapt in turn. Expert teachers such as Dr Weimer can help us with this.

should we stop students using laptops during lectures?

I guess it depends on what they’re using their laptops for.

Most days when I come in at the back of the lecture room & walk down to the front, I’ll see a lot of laptops open & in use. Quite a few students will actually have the (incomplete*) powerpoint for the day’s class open on their screens, but quite a few others are on Facebook (or some arcane form of social media that I haven’t caught up with yet) or just surfing. So when a friend shared an article titled Professors push back against laptops in the lecture hall, I read it with interest & also shared it with one of our big FB student pages for some consumer opinion. (There’s some interesting commentary here, too.)

One of the major reasons many oppose laptop use is their potential to distract students from what’s going on in the classroom, and judging from the ‘consumer feedback’ I received, that can be quite a big issue:

I don’t begrudge others using them except when they are watching videos or checking facebook etc during lectures. That’s very distracting.

It’s only annoying and distracting when people take their laptops and play games or scroll Facebook. Which a lot of people do…

Somewhat surprisingly, that distractive effect extends to students putting their devices to what many of us would regard as ‘legitimate’ use ie searching for information directly related to the class. And I’ll admit, sometimes I’ll ask a student to look something up, especially if I think they’re doing something other than class-related work! For example, this brief report cites a study showing that

students who spent a greater proportion of time seeking course-related sites recalled significantly less than those who were more often browsing sites unrelated to the course (r = -.516, p. < .02).

And worse:

the more students used their laptops, the lower their class performance (β = -.179, t(115) = -2.286, p = .024), the less attention they paid to lectures (p = .049), the less clear lectures seemed to them (p = .049), and the less they felt they understood the course material (p = .024)

Yikes! This really piqued my interest, & led me to a 2014 paper by Mueller & Oppenheimer, which has the wonderful title, The Pen is Mightier than the Keyboard. Here’s the abstract:

Taking notes on laptops rather than in longhand is increasingly common. Many researchers have suggested that laptop note taking is less effective than longhand note taking for learning. Prior studies have primarily focused on students’ capacity for multitasking and distraction when using laptops. The present research suggests that even when laptops are used solely to take notes, they may still be impairing learning because their use results in shallower processing. In three studies, we found that students who took notes on laptops performed worse on conceptual questions than students who took notes longhand. We show that whereas taking more notes can be beneficial, laptop note takers’ tendency to transcribe lectures verbatim rather than processing information and reframing it in their own words is detrimental to learning.

I’ve certainly observed that many students struggle with long-hand note-taking, to the extent that I’ll get the occasional complaint that “she moves on to the next slide before I’ve copied it all down” in my teaching appraisals. (I do explain that they shouldn’t be ‘copying it all down’…**) And I type much faster than I write, so I can sympathise with students who want to use their laptops for note-taking in class. So did some of my students, commenting that

I actually find typing notes better for me, because my typing speed is so much faster than my writing speed.

and

I would hate it if we were not allowed laptops in lectures anymore! I’d miss half the notes and then have to go home and panopto lectures (or die if they weren’t panoptoed) which just takes up time that i could use studying all my notes properly.

Mueller & Oppenheimer’s paper has really got me thinking. They point out that there is a considerable body of evidence around the efficacy of note-taking, commenting that even without the distraction effect,

laptop use might impair performance by affecting the manner and quality of in-class note taking.

This could have that negative impact on learning by two routes: ‘encoding’, and ‘external storage’. ‘Encoding’ is valuable because – ideally! – students process information as they make their notes, and doing this enhances both their learning & their ability to retain information. ‘External storage’ refers to the ability to review and learn from notes at some later point, including notes taken by others: indeed, we employ note-takers to do this for students who are unable (for a variety of reasons) to take notes themselves.

An important question here is, what are students actually doing when they take those in-class notes? Are they actively summarising what’s been discussed eg via drawing a concept map, or writing a paraphrase? Or are they simply copying, word for word, every single thing I say & show in class?*** While some could argue, “but it doesn’t matter ‘cos I’ll write a summary later”, Mueller & Oppenheimer observe that

verbatim note taking predicts poorer performance than nonverbatim note taking, especially on integrative and conceptual items.

This underlies their suggestion that while laptops allow more rapid note-taking, if those notes are verbatim, then learning and understanding may actually suffer. In fact, they observe that

One might think that the detriments to encoding would be partially offset by the fact that verbatim transcription would leave a more complete record for external storage, which would allow for better studying from those notes. However, we found the opposite—even when allowed to review notes after a week’s delay, participants who had taken notes with laptops performed worse on tests of both factual content and conceptual understanding, relative to participants who had taken notes longhand.

So where do we go from here? I must admit to being a tad flummoxed at the moment – with the need to offer more flexible learning opportunities and  the current trend to ‘paperless offices’, we’re moving into a more highly digitised world and those laptops aren’t going to go away any time soon. How, then, to overcome the apparent negative effects they may have on student learning? If part of the problem lies with the ability to take appropriate notes, do we need to somehow teach this skill to all our incoming first-years?

 

* I mean, why would I give them the whole lot up front (including the answers to my in-class quizzes)?

** no, seriously! What I’d much prefer is that they read through the material I provide ahead of class, identify the bits where they have no idea what I’m talking about, & then that’s where they should focus any note-taking during class.

*** and if they are taking such fulsome notes – how much attention is being paid to everything else that’s going on in class: the questions, discussion, extra explanations?

 

P.A.Mueller & D.M.Oppenheimer (2014) The Pen is Mightier than the Keyboard. Psychological Science 25(6): 1159-1168 doi: 10.1177/0956797614524581

riffing on the national standards

Over on Facebook, a friend of mine shared a post (from a friend of hers) about National Standards in the NZ primary education sector. If you’re on FB I recommend reading it; it certainly gave me a bit of food for thought. In his post the author, Jamie Strange, identifies what he sees as problems with the National Standards as they currently exist.

His first, that they “[narrow] the curriculum… [placing] extra emphasis on literacy and numeracy, to the detriment of other subjects”, is something that I’ve commented on previously in the context of teaching & learning in science. Back then I said that

the introduction of National Standards appears to have focused attention elsewhere, away from the delivery of science. (I know that it should be possible to address the Standards within the context of science – or pretty much any other subject – but the risk is that this won’t be recognised by many teachers without opportunities for further training.)

It would be nice to think that things have moved on in 5 years, but Jamie’s post suggests otherwise :(

Later on he states that “National Standards limits [sic] creativity in the classroom”, in terms of restricting teachers in the methods they use to help learners gain mastery. At a time when there is increasing use of innovative teaching techniques in tertiary classrooms, it would be a pity if we really are losing that at the other end of students’ learning experiences. There’s a fascinating interview with educator Sir Ken Robinson in which he discusses why creativity is something that we really, really need to foster.

And he quotes the Labour Party’s education spokesman, Chris Hipkins:

A conformist model of education that says every student has to achieve an arbitrary set of ‘standards’ at a set time in their life, will rob us. Greatness doesn’t always follow a conventional path. Students certainly need to know how to read and write, but they also need good levels of communication, self-management, perseverance, curiosity, and social skills. What can easily be measured must not become the sole measure of success.

This is expanding on something that Hipkins said in 2014:

To thrive in the 21st century, today’s students will need to leave school with a set of skills and knowledge that are quite different to what our education system has been focused on in the past. Far from ‘standardisation’ we need to focus on fostering:

• Creativity and innovation: New Zealand is a land of boundless potential, to realise that we will need to think outside the square, try new things, and take a few risks.

• Adaptability and flexibility: Look at how much the world has changed in the past 15 years. We can’t even imagine how it will change over the next 15 years and yet that’s the world those starting their educational journey today will step into. Equipping them with the skills they will need to adapt to whatever life throws at them is one of the most significant gifts we can give them.

• Collaboration and cooperation: When they step out of the education system and into the workforce, today’s students will be expected to work in teams, to problem solve, to self-motivate, and to manage their own time. Our education system needs to embrace those characteristics.

And he’s right. And his words apply to the tertiary sector as well. While ‘subject knowledge’ will remain an important attribute for uni graduates too, what one might call competencies & capabilities are just as important. These are attributes that we should foster in everyone, no matter where they’re at in their journey through our education system.

how do we assess teaching quality?

Way back when I was a secondary teacher, & there were signs that the government of the day was looking at a possible move to performance pay, there were fairly frequent staffroom discussions discussions around how to assess the quality of one’s teaching. (There’s a much more recent report on this subject here.) One metric proposed was how many of your students passed School Cert. (I told you it was a long time ago!) That was all very well for those whose classes – we had streamed classes at my school – contained students who could mostly be expected to achieve rather well. I had one of those, but I also had the ‘problem’ 4th-form (year 10) class: kids who for a variety of reasons weren’t viewed by many as likely to pass.

I had no problems with that class. I had to teach them science, and so we ‘did’ science in contexts that they found engaging & relevant: the science of cooking, the science of cosmetics, & so on. We had a ball, & in the process they seemed to absorb some knowledge of science: what it was, & how it worked. But mostly they still didn’t attempt School C (the equivalent of today’s NCEA Level 1), & so by that rubric I’d have been judged a poor teacher. Perhaps, if we’d looked systematically at the level of prior knowledge those students entered my class with, and assessed the gains they made on that, both they and I would have been judged differently.

I was reminded of this during a discussion today about assessing the quality of teachers in a university setting. Now sure, we have a system of paper appraisals and teaching appraisals. But they aren’t shared with line managers as a matter of course, and so that can make things difficult during goal-setting and promotion rounds. For in the absence of that information, just how do line managers (& others) come to any evidence-based assessmentof a teacher’s abilities and performance in the classroom? I suspect the short answer is that they can’t, not really.

But even where the appraisal data are available, they shouldn’t be the only tool individuals (& managers) use to assess performance. I’m often told the appraisals are easy to ‘game’, although I’m not sure how correct that is; it does tend to assume that students aren’t able to assess papers and teacher performance reasonably well. I mean, statements like “this teacher made it clear what was expected of
me”, “this teacher made the subject interesting”, and “this teacher was approachable when advice or
help was required” are fairly objective, after all. But ideally they’d be just one element in an educator’s portfolio.

That portfolio could also include notes and commentary from an option that teachers in the compulsory sector will be used to: having a colleague sit in on a class and provide constructive feedback afterwards. In my experience this is rare in universities, which is a real pity, because both parties can learn a good deal from the experience. (We are accustomed, and encouraged, to have others cast a critical eye on our research outcomes, so why not our teaching?)

It could also include notes & reflections from the education literature. I firmly believe that while my teaching has to be informed by current research in my discipline (& I simply can’t imagine teaching the same thing, year after year!), it must also be informed by findings from research into pedagogy.  Things change, after all. Teaching & learning methods that might have seemed to work for those who taught me at uni are almost certainly out of date in today’s classrooms. As regular readers will know, I put much of my own reflection into writing these blog posts: the blog makes up a largish part of my own portfolio.

And of course, if you’re dipping into the literature, and attending seminars or workshops from your equivalent of our Teaching Development Unit, then you’ll pick up all sorts of other, informal, tips for gaining feedback on how things are going in the classroom. It’s worth linking back to a guest post from a my friend & colleague Brydget, as she summarises all this very well.

The trick, of course, is to work out how to present that information to one’s line manager :)