thoughts on the proposed changes to NCEA

This post was first published on the Bioblog.

Many readers will probably have read this RNZ article (or heard the related interview), or seen calls for consultation on the Ministry of Education’s suggested changes to the number of subjects – and achievement standards – on offer to year 11 students.

I’ve been following (& participating, where I can) all this with colleagues and friends, and thought I’d share some of my thoughts here. But before I get onto that, I’ll point out that there’s been a fair bit of consultation even before we got to the point where these materials have gone out, in their turn, for feedback. That process began in 2018 and resulted in a “change package“. This was published in May 2019, and I really recommend reading it carefully as it provides the rationale for the latest 2 rounds of consultation (about the draft L1 Science standards & their supporting material, and about the number of individual subjects that should be offered to year 11 students.

In the interests of full disclosure, I’m a member of the Subject Expert Group (SEG) that is working on the draft L1 Science achievement standards.

So, the SEG members were tasked by the Ministry with developing four Science achievement standards (ASs), but that decision on the number of standards was based on a lot of feedback from a wide range of sector & interest groups, which signalled very clearly a need to reduce the complexity of NCEA & reduce the number of standards¹.

I’ll admit that one of my concerns regarding these two recent consultation rounds is the overlap between requests for feedback about the initial drafts of the Science material, and the announcement of consultation on the number of subjects on offer. I think it’s meant that people have conflated the two.

But – none of this is set in stone; it’s all draft material. Feel strongly about it? Then follow the appropriate links above, and be heard. And – read all the relevant materials before you comment.

One of the things I’ve heard quite often about the Science ASs is that the actual subject material is “hidden”. To some degree this might be due to people reading the headlines, and the ASs, and not also going through the supporting material: the learning matrix (which clearly identifies content) or the Teaching, Learning & Assessment Guide (TLAG for short). But from my perspective, the content material for biology, physics, chemistry, and earth & space science remains the same, and provides an essential context for delivering concepts and competencies relating to the Nature of Science strand in the National Curriculum document (NZC). Hopefully the next round of consultation documents will see the inclusion of some examples of teaching and assessment plans that show what this would look like in practice.

Thus, I think there does need to be an element of trust that teachers will continue to deliver content, & in fact – speaking personally – I would hope there will be a clear statement at some point about the need to cover content. However, I also think it’s important to remember that at the moment there are 31 standards available to schools delivering a year 11 Science program (which is almost all of them) and thus there is no guarantee of consistency now about what content students may or may not have covered.

I’ve heard a lot of concern about the need for professional learning development (PLD) opportunities for teachers. It’s a concern that I know is shared by all of us on the SEG, and it’s one that we’ve communicated to the Ministry. This is a shift in direction; it will entail a significant amount of work by classroom teachers; and there absolutely needs to be a substantial amount of PLD available well before implementation of any confirmed changes to the NCEA. (Not least, for science teachers, because the year 11 changes will probably flow down – to year 9 & 10 classrooms – and may have some impact ‘upwards’ as well.

But – & it’s a very big ‘but’ – I think that it would be easy to lose sight of the fact that the proposed standards are very much aligned to the NZC in placing  the nature of science front & centre (its delivery to date, if present, has been largely implicit).  As I wrote in my previous post,

Back in 2007 New Zealand implemented a new national curriculum. One of the features of the science component of that document is the overarching importance of students gaining an understanding of the nature of science (the “unifying strand” of the curriculum). In that context, it expects that:

students learn what science is and how scientists work. They develop the skills, attitudes, and values to build a foundation for understanding the world. They come to appreciate that while scientific knowledge is durable, it is also constantly re-evaluated in the light of new evidence. They learn how scientists carry out investigations, and they come to see science as a socially valuable knowledge system. They learn how science ideas are communicated and to make links between scientific knowledge and everyday decisions and actions.

And the document specifically adds that these outcomes are pursued through the following major contexts (the various science ‘subjects’) in which scientific knowledge has developed and continues to develop.

 

Given that currently about 60% of students in year 11 science don’t go on to further study in any of the sciences, I’d argue that while a scientifically-literate society does need some knowledge of science, it also requires a solid understanding of the nature of science itself.

 

 

¹ In my personal opinion, the inclusion of additional specific subject standards at year 11 would pretty much destroy the kaupapa of the SEG’s work, in that we would not see students gaining that key, core understanding of NoS. The nature of the 4 ASs currently out there for feedback was not determined randomly, but as the result of a fair bit of thought and discussion by the SEG members.

why do students need to learn about the nature of science?

This post was first published on the Bioblog.

You’re probably aware that the Achievement Standards used to assess senior school students’ learning are being reviewed. Science is one of the ‘pilot’ subjects in this process, where a ‘Subject Expert Group’ has developed 4 draft Science standards¹ (a significant step away from the current 30+, and a response to advice from several high-level advisory groups). These drafts have been out for consultation, and are all intended to develop and assess students’ understanding of the nature of science, with subject content providing the contexts for this learning. (That is, the subject content has definitely not disappeared.)

Why is this important?

Back in 2007 New Zealand implemented a new national curriculum. One of the features of the science component of that document is the overarching importance of students gaining an understanding of the nature of science (the “unifying strand” of the curriculum). In that context, it expects that:

students learn what science is and how scientists work. They develop the skills, attitudes, and values to build a foundation for understanding the world. They come to appreciate that while scientific knowledge is durable, it is also constantly re-evaluated in the light of new evidence. They learn how scientists carry out investigations, and they come to see science as a socially valuable knowledge system. They learn how science ideas are communicated and to make links between scientific knowledge and everyday decisions and actions.

And the document specifically adds that these outcomes are pursued through the following major contexts in which scientific knowledge has developed and continues to develop.

The development of that list recognised that the country’s future prosperity depends on students continuing to study science and entering science-related careers. This is because – as the late Sir Paul Callaghan observed –‘rich’ countries depend on high-end science and technology, and NZ needs to invest far more heavily in these fields to maintain and enhance its standard of living. That is, we need more scientists, scientifically-literate politicians, and a community that understands what science is done and why it’s relevant to everyday life.

But in practice, since then we’ve probably focused more on subject content than on explicitly teaching what science is, how it works, why it is such a powerful tool for understanding the world around it, and that it is a human/social endeavour. (I’m sure it’s implicit in many programs, but things like this aren’t universally picked up by osmosis: practice reinforces learning.)

Does this matter?

Well, yes it does. Knowledge of content is important, but I’d argue that it is far from being enough. Around 60% of year 11 (NCEA L1) students won’t go on to take science subjects at year 12 or 13. They need – all students need – more than content to be science-literate (as this recent PISA document makes clear). To that end, the NZ Curriculum document asked that in addition to content knowledge, students gain the ability to critically evaluate science ideas and processes; to communicate about science; and to recognise that science is a human endeavour² (people develop our scientific knowledge and that their ideas change over time).

And having the knowledge, understandings, and competencies that should be delivered by a teaching & learning program assessed using these standards, students should then be able to critically engage with the various science-based & science-informed issues that they’ll encounter, now & in the future. (And to deal with claims such as “well, science got it wrong in the past, so it can’t be trusted now”; and “science is always changing its mind”, both of which are hallmarks of those arguing against established scientific knowledge.)

That’s what the draft standards are intended to deliver, together with the acquisition of content knowledge. And I think that’s a very good thing.

 

¹ disclosure: I am a member of this group.

² The concept that science is a human endeavour is explicit in the title of one of the draft standards.

the sad state of science learning in primary school

This post was first published on my ‘other’ blog. It’s not intended to diss primary school teachers – quite the reverse! They need all the help & support they can get to help them deliver the science curriculum.

In 2011, Sir Peter Gluckman released his report, Looking ahead: science education for the 21st century. In it, he noted the need to improve science teaching in primary schools, commenting that

there should be an attempt to improve the confidence [my emphasis] of all teachers within primary schools to assist in science and that all primary schools should be encouraged to develop a science champion.

And in 2012, David Vannier pointed out that

there is growing evidence that too many children are not doing well in science and do not have access to effective instruction, especially at the primary level.

and that

[at] the same time that the New Zealand government is seeking to spur innovation in science as a means to improve the economy, less and less emphasis is being placed on science instruction in primary schools.

Fast forward to Monday this week, when Radio NZ reported on the findings of The National Monitoring Study of Student Assessment (NSSA): that 20 percent of Year 8 children last year reached the expected level of achievement in science – the lowest figure of any learning area in the curriculum. While most children liked learning about science at school – 82% of those in year 4 and 65% in year 8 – those figures haven’t changed significantly since the previous survey in 2010, and the decline between years 4 and 8 should be a concern. Overall, these results don’t augur well for science literacy and engagement with science amongst our young people.

You may be tempted to lay this result at the feet of National Standards. Don’t. Looking Ahead was published in 2011. National Standards were first implemented in 2010, just a year earlier. The issues identified by Sir Peter Gluckman have had a longer gestation than that.

I wrote about Sir Peter’s report at the time, highlighting his statement that

science education is not just for those who see their careers involving science but is an essential component of core knowledge that every member of our society requires.

Thus, science education needs to deliver on what Sir Peter characterised as ‘citizen-focused objectives’, where all children need to have:

• a practical knowledge at some level of how things work;
• some knowledge of how the scientific process operates and some level of scientific literacy
• enough knowledge of scientific thinking as part of their development of general intellectual skills so that they are able to distinguish reliable information from less reliable information.

But can it deliver? His report also notes that

[a] well prepared primary school teacher will integrate excitement about the natural world and scientific forms of thinking into literacy and numeracy teaching, and into general educational processes. The challenge is how to provide primary teachers with the skills to do so. [My emphasis]

I believe that meeting this challenge will require changes to at least two things: teacher-training curricula, and professional development (PD) and support.

Just 25% of primary school teachers hold another qualification, in addition to their teaching degree, and it’s probably fair to say that BSc graduates are in a minority. Intending primary school teachers usually study for a 3-year Bachelor of Teaching degree, and take a range of papers in their first year – including one on science teaching. This one paper, plus learning opportunities while on practicum in schools, may well be their sole exposure to science (Campbell, 2018).

Which is where the PD and support come in. Ally Bull (2016) found that science was “marginalised” in the primary curriculum; and that teachers – lacking confidence to teach the subject – often had little in-school support and only limited access to opportunities for PD. The majority of those providing the PD (51%) aimed to enhance teachers’ confidence to teach science, and just 5% felt that developing their knowledge of science was important. Bull also cited other research that found that “primary teachers’ lower confidence in low confidence in teaching science reflected their lesser degree of content knowledge.”

There are ways to address this. Anne Hume & Cathy Buntting (2014) developed resources and shared these with primary teacher trainees, encouraging them to think about what science ideas they could teach (plus the why, when & how) while using those resources. Their results? Really encouraging:

Even student teachers who had previously felt very apprehensive about teaching science reported feeling far more confident about the prospect after completing the CoRe assignment.

Programs like theirs, changes in teacher education, and the commitment to provide ongoing mentoring and support, should raise teachers’ confidence in teaching science and see them reach their full potential as ‘science champions’. Our teachers and our children deserve no less.

 

 

teachers’ reactions to this year’s year 13 bio exam

Today I’ve been hearing from some very unhappy teachers. As in, teachers who are upset to the point of tears on behalf of their students. Excellent, very experienced teachers. The reason for their unhappiness? This year’s NCEA Level 3 (year 13) biology exam, sat by their students just a few days ago.

And at this point I should emphasise that the teachers’ concerns were focussed towards the New Zealand Qualifications Authority, and not the individual examiner(s) who, after all, prepare these documents with advice and guidance from NZQA staff. Their concerns were focused on the system.

Now, it’s several years since I was an examiner at this level, and I know that the nature of the exam has changed. And of course the teachers themselves are well aware of what’s been expected in the past; they’re just taken aback by the nature of this year’s papers**.

Thing is, I was also involved in developing Scholarship-level exams and to me, while for some questions there’s quite a bit of resource material to get through in this year’s L3 exam, the amount of writing required of students seems an awful lot for level 3. The question-books for the exam (you’ll find them here) contain multiple blank pages for students to write their answers: the implicit message is that a lot of writing is needed. There are 3 questions like this in a book, so three essay-type answers for students hoping to achieve an excellence for the paper.

This may not sound like much – but the actual exam covers 3 separate achievement standards. So a student who’d prepared for all three (and most schools encourage this) would find themselves faced with writing up to 9 (yes, nine) extended answers over the space of three hours. (For comparison, a Schol Bio candidate would write just 3 essays in the same time frame.) In other words, the demands of this exam are such that it would quite likely preclude students from doing justice to all three papers.

So, here are some of the teachers’ concerns (I’m quoting with their permission):

• for an ‘excellence’ response, a student had to demonstrate high-order analysis & evaluation skills, in an answer generated in just 20 minutes (less, really, because of the requirement to read the question & plan an answer first). This is a big ask.
• students who were slower writers, or who had lower (but still OK) literacy levels would struggle to complete in a way that still allowed them to demonstrate their knowledge about biology and so gain an ‘achieved’. [And let’s remember that there’s a lot more to knowledge than simply being able to write a bunch of definitions.]
• “all my students felt let down by this examination. All their hard work, dedication, and love for the subject were lost in those 3 hours.”
• “I have seen a fair few of my students this week and they are so demoralised! Many are gutted they tried to do all 3 papers [because they] couldn’t do them justice- am gutted for them!”
• “A Facebook comment stated, “OK, I will play their game and only do two externals next year”. What is happening to the integrity of our subject when the assessment is driving the whole course structure of Biology in schools throughout the country?”
• “We are losing students because the subject is deemed too difficult. A colleague informed me that her daughter would not be taking Biology next year because it was too hard so she is taking Physics and Chemistry instead.”

Yes, this is anecdotal. But if these comments reflect a widespread reality, then science education in this country will be the loser.

 

** I was rather concerned about a particular question, too – but that’s best left to another post, on my ‘other’ blog.

talking about what we should teach

This is a cross-post of something I originally wrote for my ‘other’ blog.

While I was on holiday (Japan – it was wonderful!) – I read Tom Haig’s interesting article about ‘curriculum wars’ over on Education Central, and it reminded me of the concerns I’ve held for some time that we don’t really talk enough about what to teach in our classrooms, be they university-level or in the secondary sector.

Several years back (how time flies!) I was involved in developing the ‘Living World’ component of the New Zealand Curriculum document, as well as entering into the discussions around what the science component of that document should deliver. (Right down to a discussion of what it actaully is to ‘do’ science.) At the time I was somewhat taken aback to discover that the panel was not required to give any exemplars for teachers, any indication of what they might do to help students master particular concepts – something that’s noted by Tom. Yes, I totally get it that schools are free to set their own curricula, but at the same time I couldn’t halp thinking that the occasional ‘starter for 10’ might be useful.

Layered on top of that – & amplified by my experiences in relation to developing and assessing Achievement Standards for NCEA, was the way that while new content or concepts might be loaded on up-front, we didn’t seem to remove stuff at the other end. This had the result that the amount of information associated with a standard might just grown & grow (CRISPR, anyone?). Pretty much the same thing tends to happen at university – if you look at one of the standard first-year biology textbooks, Campbell Biology^A, you’ll see that it’s become steadily thicker over time as new material’s added. (In my experience, at least some first-year uni lecturers argue that all the basic stuff should be delivered at school; they shouldn’t have to teach that. However, this sits poorly against the fact that no NZ universities have any prerequisites for their first-year biology papers, and also suggests that those making the statement don’t really recognise that not all year 13 students are heading for university. Remember, schools have the ability to shape their curricula to suit the needs and requirements of their individual communities.)

In other words, we didn’t seem to be having any discussion around what should be taught, and why. And we still don’t, although hopefully such issues will be addressed in the review of NCEA. For, as Tom Haig says:

Working out what we should be teaching, and why, is something that we should be discussing together and taking much more seriously as teachers than the second place it’s taken to discussions of technique. Hattie, ERO, the Best Evidence Synthesis and so forth are filled with advice about ‘how’, but shouldn’t we be thinking just as hard about ‘what’?

^A No relation! I was privileged, though, to meet the late Neil Campbell when he visited New Zealand, and was struck by what a wonderful educator he was.

what are the challenges for first-year core science courses?

This is another post based on a talk at FYSEC2017, & which I’ve also published on my bioblog.

Prof Karen Burke da Silva was the keynote speaker at Day 1 of the 2017 First-Year Science Educators’ Colloquium, held in Wellington. Her topic:Transforming large first year science classes: A comprehensive approach to student engagement. Currently at Flinders University, she’s been instrumental in setting up an ‘integrated teaching environment’ that’s seen a drop in withdrawals, and a marked increase in engagement, among their first-year STEM students.

If you’ve read my earlier FYSEC-focused post, you’ll know that student engagement was a hot topic at last year’s colloquium. Which isn’t surprising; as Karen noted, both NZ and Australian universities have trouble with attention, engagement, retention, and performance of their first-years, who face some significant challenges in transitioning from their smaller high-school classes to the large lecture rooms of universities. She commented that

how best to build a first-year program in sciences that allows for different student backgrounds, abilities and interests is a task that all first-year coordinators face.

Because students are so diverse, if we’re going to accommodate their various needs and backgrounds, we really need to know about those first. In Australia, the SSEE Project gathered data on both student and staff expectations and experiences (& whether the two converged) across all disciplines at Flinders, the University of Adelaide, and the University of South Australia. The decision to set this research project up was based on some reasonably concerning information:

• Statistics show that of all students entering Australian universities one-third fail to graduate and of those students who withdraw from their programs over half withdraw in their first year.
• Students preparing for tertiary study may do so individually or via school, government and university initiatives. Many students, however, still experience an early ‘reality shock’ during their first semester rather than a smooth transition to university.
• The mismatch between students’ expectations and experiences has ramifications for their learning, satisfaction, retention and ultimately, their wellbeing.

Among the findings that Karen presented to us:

the majority of students were neutral about, or agreed with, the statement that secondary school education was an adequate preparation for university study;

students from schools offering the International Baccalaureate program outperformed those from all other schools on entering university (with students from state schools doing least well), & the difference was still reflected in GPAs at the end of that first year:  However, the difference between state and private schools disappeared over that time;

friends, university websites, and universities’ recruiting efforts had more effect on shaping students’ views about university study than teachers, guidance counsellors, family, new and traditional media outlets, and provided a more accurate reflection of what uni life is really like.

students’ expectations around what constituted a reasonable time interval for returning marked work to them were not matched by the reality: the majority expected it back in 2-3 weeks, but in reality most waited 3-4 weeks;

the great majority felt that receiving feedback on drafts would be very important to their learning – but most disagreed with, or were neutral about, the statement that they actually received such feedback. (While students may not be aware that there’s more to feedback than written comments on an assignment, providing feedback in a timely manner is something that most universities need to work on.)

When Karen arrived at Flinders, back in 2007, the STEM disciplines had a high fail rate of around 23%; this was particularly noticeable among mature students & those who hadn’t taken the final year of high school. The changes she & her team made to teaching delivery were intended to address this, but they would have the effect of enhancing the learning experience for every student. I found her ideas around this really exciting (although I suspect that those wedded to a more ‘traditional’ approach to delivery would be shaking their heads).

This is what the new program looked like: first up, the first semester of the year became ‘transitional’, ensuring that everyone was in the same place before entering semester 2, which was ‘extension’, taking students’ knowledge & understanding further. Along that were ‘pre-lecture’ classes^A for students identified as lacking the normally-expected background in the subject, which resulted in the students having greater confidence in their ability to cope with the subject, plus increased motivation & understanding. And I loved  the idea of regular case-based ‘lectorials’, where the students were actively engaged in addressing the issues raised in each case study. Karen’s research showed that 98% of students reported that these classes enhanced their understanding of how biology relates to the real world.

Learning was further supported by peer-assisted study sessions, run by 2nd- & 3rd-year students (who received training for the role), which were part of the formal timetable and for which students could gain up to 5% of their final grade for attendance. Karen reported that these sessions were very well attended.

And of course, STEM subjects have labs. Karen told us that Australian universities are tending to reduce the lab component of STEM papers, such that most first-year papers have less than 30 hours of practical classes – this is a real pity as in general students really enjoy labs and the practical classes (if properly focused) can enhance understanding of key concepts as well as teaching a range of practical skills. (I’m often perplexed by suggestions that we move to on-line ‘labs’, as both lab & field work have a lot of practical & interpersonal skills development associated with them, & that’s something that you don’t get by interacting with a mouse & a screen.) At Flinders, Karen told us that a science paper would have 2, two-hour, lab classes each fortnight: the first session is all about preparation & planning, & the second is the actual practical work. It seems to me that this would give students a good experience of actually ‘doing’ science – something that the students agreed with, as well as reporting that they liked becoming more responsible for their own learning.

The research projects that all science students at Flinders do in their first year of study would also have that effect, although they have to be scaffolded into these assignments – which also provide an excellent opportunity to learn many of the personal skills needed for successful teamwork. (This is another of those competencies that universities often say their students gain, but for which they often don’t really provide much in the way of carefully-designed learning opportunities.)

I was fascinated to hear that Karen also includes art, & other creative tasks, in her assessment tools – this is great as it allows students to recognise that science contains an element of creativity. She commented that having the first assignment as an art project both helps to remove the fear associated with doing a science assignment, and helps connect the teacher with their students. The question she sets is a very simple one: what does biology mean to you? These were self-graded, something that would make many science lecturers raise their eyebrows! – but apparently in moderating the results Karen’s found that 90% of the class awarded themselves the same marks that she would. Of the remaining 10%, those who graded themselves lower tended to be female, while those giving a higher mark were male. The students submitted some amazing work.

Apparently other staff weren’t always happy as they felt that students didn’t give their own assignments the same attention – but there was a happy outcome: they began to look at ways of offering the opportunity for similar assignments, with a real-world focus, in their own papers. I’d do that myself, given that these changes in delivery & assessment had a marked impact in terms of student outcomes, with fewer failures & withdrawals.

And we were reminded that students need to feel some connection with the institution & with those teaching them. (There’s quite a lot of literature available on this, including TLRI studies from NZ & other papers like this.) Having that contact offers opportunities to find out how the paper is progressing, & also to identify any problems that students might be having & to refer the students to appropriate support if necessary. I think it would also help lecturers to understand the school system that our students have come from; having that understanding is crucial in optimising the transition from secondary to tertiary learning environments.

We ended with some questions around the value of recording lectures. My institution does this; I suspect most universities in NZ do. Feedback from students indicates that the practice is helpful for international students, those wanting to review their understanding, & for those who’ve had to miss a class; Ican certainly see the peak in views just before a test! But we’re finding that many students neither attend class, nor view the recordings, & while some may muddle through like this, others don’t. So, we need to come up with a way to change students’ mindsets – and for their seemingly insatiable demand for recordings & lecture notes & previous exams. (This is something that’s definitely a carry-over from school, I think.) So, how do we deal with that demand, that sense of entitlement, that lack of engagement? I’m not sure I have the answers. Do you?

Karen thinks recorded lectures have changed face of education in a very negative way. Good for internationals, for high-achievers, for review. But the mid-range group don’t show, don’t view the recordings either. If we’re to continue with recordings then we need to change the student mindset as well.

^A For those interested in the concept of prelectures, here’s the abstract from one of Karen’s papers on the subject:

First year biology students at Flinders University with no prior biology background knowledge fail at almost twice the rate as those with a background. To remedy this discrepancy we enabled students to attend a weekly series of pre-lectures aimed at providing basic biological concepts, thereby removing the need for students to complete a prerequisite course. The overall failure rate of first year biology students was lowered and the gap between students with and without the background knowledge was significantly reduced. The overall effect of the implementation of pre-lectures was a more appropriate level of teaching for the first year students, neither too difficult for students without a prior biology background and no longer too easy (or repetitive) for students with high school level biology.

engagement & experiences in undergraduate science education

This post is based on a presentation at the 2017 First-Year Science Educators’ Colloquium (FYSEC), and is also published on the Bioblog. 

At FYSEC2017, Gerry Rayner led a session called “Undergraduate science education in the 21st century: issues, needs, opportunities”.

Gerry kicked off by commenting that education has a greater impact – on students, teachers, and the wider society in which education systems are embedded – when people work together across a range of disciplines. What are the issues currently facing undergraduate science in NZ & Australia, he asked, and how do we address them? This was something that generated quite a bit of subsequent discussion. On the list:

• rising enrolments: Gerry commented that in Australia, the removal of caps on enrolment, together with international demand, meant that some predictions of student numbers saw growth of perhaps 30% over the next few years’
• increased diversity – not only cultural and ethnic diversity, but also a wider range of prior knowledge and academic achievement on entry;
• as fees increase, and with that, student debt, we’re already seeing a change in attitude: students see themselves as customers, paying for a product, and can expect particular outcomes;
• lower on-campus attendance may well have an effect on student engagement (and comments from attendees showed that this is something we all face) – but, to support increased numbers, we are pushed to provide more on-line delivery;
• this means that educators need to provide not only more on-line content and assessment, but also the sort of meaningful interactions that enhance student engagement;
• the need – Gerry described it as a moral obligation, & I agree that the obligation is there – to provice meaningful opportunities for students to enhance their employability. That is, it’s not all about mastery of content, and students also need to gain a whole range of work-related competencies and capabilities.

Gerry then introduced some data from a report on student engagement in New Zealand universities (Radloff, 2011), which defines this thing called ‘engagement’ as

students’ involvement with activities and conditions that are likely to generate high-quality learning, [something that] is increasingly seen as important for positive learning outcomes

and comments that

measures of student engagement provide information about individuals’ intrinsic involvement with their learning, and the extent to which they are making use of available educational opportunities. Such information enhances knowledge about learning processes, can be a reliable proxy for understanding students’ learning outcomes and provides excellent diagnostic measures for learning enhancement activities.

This wide-ranging report is based on data from the AUSSE^A survey of student engagement, & includes chapters on Maori and Pasifika student engagement; engagement in relation to field of study; the experiences of international students; relationships between engagement, preparation for study, and employment; students’ departure intentions; differences between part-time & full-time students; and the impact of distance education cf on-campus learning on student engagement. The survey has 6 engagement scales (academic challenge, active learning, student/staff interactions, enriching educational experiences, supportive learning environment, & work-integrated learning), & 7 outcome scales (higher-order thinking, general learning outcomes, general development outcomes, career readiness, average overall grade, departure intention, and overall satisfaction). In Radloff’s report the AUSSE data from NZ were also benchmarked against responses from Australian, South African, and US undergraduate students.

The results, said Gerry, were generally good but (& the report also makes this clear) not entirely comforting. In measures of engagement, for example, NZ students rated the quality of staff-student interactions quite poorly (an average score of 18 compared to 35 in the US); and a low proportion (across all countries) felt that they had enriching educational environments – while at the same time strongly agreeing that they had quite a supportive learning environment!

And on the ‘outcomes’ scales, only about a third of NZ first-year students felt that they had gained some level of career readiness through their uni studies. At the same time, around 30% of them had considered leaving university (yes, there were a range of reasons underlying this). Even by the end of the degree only 35% felt that they were really career-ready, & 29% had considered leaving during the year. This is not particularly positive.

Overall, for the natural & physical sciences, NZ students felt that: they didn’t get a lot of support from their university; they were less likely to answer questions or get involved in discussions; they had low levels of interaction with others in their class; felt they had lower career readiness, and lower levels of workplace-integrated learning experiences, than students from other disciplines (in fact, in this 2011 report only 9% reported involvement in some sort of placement or work experience); tended to have jobs unrelated to their future study/career hopes; and were less likely than those from other disciplines to feel that their study at uni helped prepare them for the workplace.

And again, there’s that 30% of them who either considered leaving, or planned to leave, before completing their studies (but those reporting working regularly with others in class were much less likely to be in this group). However, it’s not all doom & gloom on that front:

while nearly one-third of New Zealand’s university students have seriously considered leaving their university before completing their study, students are generally very satisfied with their experience at university. [Around 75%] rated the quality of academic advice received as ‘good’ or ‘excellent. [And more than 80%] were satisfied with their overall educational experience… The vast majority … indicated that given the chance to start over, they would attend the same university again.

Nonetheless, Gerry argued (& I agree), it appears that as a country we don’t prepare science students particularly well for the workplace – despite the fact that we’d hope that they will be contributing to the ‘knowledge economy’. So the delivery of workplace-integrated learning (WIL) becomes something that STEM faculties need to look at more closely. We also need to work on improving student perceptions of the nature of their learning experiences & outcomes. Here, Gerry suggested that experiential learning that helps develop skills as well as content knowledge, peer tutoring, innovative use of technology, case studies, group work, and role playing can all help – and can also be a part of preparing students for the WIL component of their learning, and for the workplace after university. (Of course, this means that institutions also need to provide ongoing PD for their teaching staff, to support them in using new means of delivery.)

Students benefit from WIL, as they can get a better understanding of the world beyond the universities. This is true even for projects run on campus, so long as there are industry links of some sort and the students are working on authentic problems that let them apply their content knowledge in real-world contexts. But WIL has benefits for academics as well, as the improved connections with employers can deliver research opportunities. It requires effort (& investment) to set up, but the outcomes for institutions and students would make this worthwhile.

^A AUSSE: the Australasian Survey of Student Engagement

A.Radloff (ed.) (2011) Student engagement in New Zealand’s universities. pub. ACER & Ako Aotearoa. ISBN 978-0-473-19590-8

what’s feedback – and do unversities do it well?

This is something that I’ve also posted on my ‘other’ blog, based on a most excellent article that a colleague has just sent me.

I’ve just received a reminder that I need to set up the paper & teaching appraisal for my summer school paper. This is a series of items that students can answer on a 1-5 scale (depending on how much or how little they agree with each statement), plus opportunities to give open-ended responses to a few questions. These last are the ones where I might want to find out how the students think I might improve my teaching, or the aspects of the paper that they did & didn’t like.

Among the first set of items is usually a stem along the lines of “this teacher provides useful feedback on my work”, where responses would range from ‘always’ (1) to ‘never’ (5). It’s the one where I get my lowest scores – and this is despite the fact that I provide general feedback to the class, written individual feedback on essays etc (& when I was teaching first-year, the opportunity to get feedback on drafts), and verbal feedback when the opportunity is there. Digging into that a bit, it appeared that most students only saw the written feedback as feedback at all, and since a substantial minority didn’t collect their essays afterwards, then they felt they weren’t getting feedback. Bit of a catch-22, and one that perhaps marking & giving feedback on line might ameliorate? I hope so.

But you can understand why students might not participate in an appraisal of the paper and the teaching in it: if they feel that the teachers aren’t providing them with feedback, why bother? And – just as important – if we don’t close the loop & tell students how we use their feedback, then why would they bother?

So, are universities good at providing feedback to students? I don’t agree, and I think quite a few students would say no – and according to this excellent article in the Conversation, academic researchers, Australia’s 2015 Graduate Course Experience survey, and the Australian government’s “Feedback for Learning” project agree with them. For example:

The 2015 Graduate Course Experience surveyed over 93,000 students within four months of their graduation. It reported that while close to three quarters of graduates felt the feedback they received was helpful, 16.3% could not decide if the feedback was helpful, while a further 9.7% found the feedback unhelpful. Clearly something is wrong when a quarter of our graduates indicate feedback is not working.

The findings from the Feedback for Learning survey of more than 4,000 students are particularly interesting – & saddening. Of all those surveyed, 37% said that the feedback is discouraging. Thirty-seven percent!!! There were few instances where students felt that they’d received the opportunity to benefit from any formative feedback they received. 15% of all respondents found the feedback upsetting – but this rose for international students, students with poor English skills (these first two are not necessarily one & the same) or a learning disability. And a majority of both staff & students felt that the feedback is impersonal.

You can see why I found the article saddening. But why is there such a problem? Perhaps, suggests the Conversation, it’s partly (largely?) because in many cases both academics and students don’t really understand what ‘feedback’ really is.

For example, many academics and students assume that feedback is a one-way flow of information, which happens after assessment submission and is isolated from any other event. In addition, academics and students often feel that the role of feedback is merely to justify the grade. A further misunderstanding is that feedback is something that is done by academics and given to students. These beliefs are deeply held in academic culture.

Luckily there are things that we can do about it. The article describes four things that educators should bear in mind that would significantly improve both the quality of feedback that we provide, and the nature of students’ learning experiences arising from that feedback. I strongly recommend reading those recommendations – and acting on them.

more on laptops in lectures

I type much more quickly than I write (some would argue, also more legibly). But when I’m taking notes in meetings, I do it with a (very old-fashioned) fountain pen & notebook. The reason is that this makes me filter what I’m writing, so that only the relevant points make it onto paper.  And this is why I’m actually somewhat chary of requiring, or expecting, students to take lecture notes on laptops, despite the push in many quarters for ‘bring your own device’ (BYOD) to classes in the expectation that students will do just that.

Yes, there are some good things about using laptops in class (see here, for example – it’s a commercial site but I ignored the little pop-ups wanting to sell me things). They allow for faster note-taking, & if students are using google docs for that, then they can access their notes anywhere – they can also collaborate on the notes, which offers some exciting possibilities for peer-assisted learning. Laptops & other devices can also increase engagement eg via using them to complete in-class quizzes & polls.

However, they also allow for people to feel that they are multi-tasking – tweeting (as many academics do at conferences these days), chatting on messenger, posting on Facebook. Unfortunately that means that their attention’s divided and their focus on learning is diminished. It could be – and has been – argued that that’s the educator’s fault; that we should offer such engaging classes that no-one’s interested in goofing off, and indeed I think there is some truth in that. After all, if what the lecturer says is pretty much identical to what’s in the slides they posted on line, many students may not see much incentive to pay attention, because “I can always read the notes or watch the recordings later”. (Only, many never do :( )

What’s more, the off-task use can be distracting to other students as well as the individual users:

We found that participants who multitasked on a laptop during a lecture scored lower on a test compared to those who did not multitask, and participants who were in direct view of a multitasking peer scored lower on a test compared to those who were not. The results demonstrate that multitasking on a laptop poses a significant distraction to both users and fellow students and can be detrimental to comprehension of lecture content (Sana, Weston & Cepeda, 2012)

and

Most importantly, the level of laptop use was negatively related to several measures of student learning, including self-reported understanding of course material and overall course performance (Fried, 2006)

and

Results show a significant negative correlation between in-class phone use and final grades… These findings are consistent with research (Ophir, Nass, and Wagner 2009) suggesting students cannot multitask nearly as effectively as they think they can (Duncan, Hoekstra & Wilcox, 2012).

Laptops & tablets also allow for very rapid note-taking – and yes, I’m saying that like it’s a bad thing. But if you’re typing so quickly that you can take down what’s being said verbatim, then you’re probably not processing the information, and that has a negative effect on learning and mastery of the material further down the track. This was investigated by Mueller & Oppenheimer (2014), who found that even when students were completely on task i.e. using their devices only for note-taking, their engagement and understanding was poorer than those taking notes longhand. (That’s in addition to other negative impacts they identify: students off-task, poorer academic performance, and even being “actually less satisfied with their education than their peers who do not use laptops in class.”)

Mueller & Oppenheimer cite earlier work that identified two possible, positive, impacts of longhand note-taking: the material is processed as the notes are made, which improves both learning (makes it more likely that deep, rather than shallow, learning will occur) and retention of concepts; and the information can be reviewed later (of course, that’s also possible with digital notes).  Processing usually involves paraphrases &/or summaries – which is what my meeting notes generally look like – but can also involve tools such as concept mapping, and there’s a lot of research showing that students involved in this sort of activity do better on tests of conceptual understanding and the ability to integrate information.

So, since it’s those higher-order skills that we hope to develop in our students, perhaps we need to tread carefully around the BYOD idea. Or at the very least, discuss all these issues with students at the start of the semester!

 

 

unplugging a flipped classroom

The always-excellent Faculty Focus has been running a series on techniques for developing and running flipped classrooms. I’ve been reading them with interest, because – as some of you might remember – I’ve ‘flipped’ some (but not all) of my own teaching sessions.

Now, my own classes have been pretty low-tech; with the ‘design-an-organism’ classes (an idea that I learned from my colleague Kevin Gould), students are expected to do a bit of revision of their notes, but the actual lecture-room experience involves nothing more than group work + pens & paper (& a projector to share the results).  So the topic of a recent post naturally caught my eye: The Flipped Classroom Unplugged: three tech-free strategies for engaging students – not least because at my workplace there’s an increasing amount of discussion around ‘going digital’, and we need to take care not to throw the baby out with the bathwater.

Dr Barbi Honeycutt’s list includes: adapting the ‘muddiest point’ feedback technique (except now it’s the students who analyse the comments for commonalities and patterns); mind-mapping; and a brain-storming challenge.

I use mind-mapping quite a bit in class, & also in my own thinking & planning. Barbi’s post reminded me of the PhD research of my friend (& then-student), Cathy Buntting, in which she had me teaching students in tutorial classes how to develop mind-maps using the same tools Barbi describes: post-it notes, pens, & large sheets of paper (in place of whiteboards). (Writing concepts on the sticky notes lets students move them around, revising their maps as their understanding changes.) We also encouraged the students to use concept maps in their revision & to plan essays in exams – and Cathy found that the sort of deep learning encouraged by this technique really paid off in those examinations: students who’d learned complex information using concept maps did much better on questions testing complex understandings than those who tended to use shallow, rote learning methods. (There was no difference between the groups when it came to rote-learning tasks.) She also found that a large majority of students thoroughly enjoyed these tutorial sessions and found the mind-mapping technique both enjoyable and helpful.

I’ve used concept mapping widely (though not exclusively; a range of tools is much better) ever since. However, in lecture classes it’s usually been as a means to show how to review knowledge of a topic & plan out an exam answer, after students have spent time in discussion. In future, I really must be a lot more active in encouraging their own use of this tool in lectures, & not just in the smaller, more manageable tutorial sessions.

Thank you, Faculty Focus!