Talking Teaching

May 12, 2014

facebook – more than just social networking

Some of my readers over on Sciblogs will probably have realised that I quite like Facebook – not least because it’s a good source of gorgeous images and quirky facts that can start me thinking about a new science blog post. (You don’t see that side of me here on Talking Teaching :D ) Also, it’s fun keeping in contact with friends & participating in various discussion groups.

One of those groups was set up by the biological sciences students at my institution, and it’s used mainly for sharing biology articles and images, the occasional in-joke :) , and alerting other students to upcoming events that their committee has organised. This particular page sees a bit more student activity than some of our paper-specific moodle pages, so for a while now I’ve wondered about the potential of a good Facebook page to be more than ‘just’ a place to hang out and share pictures & stories.

Anyway, recently I had a conversation (on FB, lol) with a couple of fellow Ako Aotearoa Academy members about this potential. It turns out that they both use FB quite extensively in their teaching lives and gave me a lot of helpful hints – along with a very recent paper on this very subject (Dougherty & Andercheck, 2014).

Kevin Dougherty and Brita Andercheck teach a large (around 200 students) introductory sociology class at Baylor University in the US. Like all those with classes of this size (or larger), they recognised that one of the major issues they face is

the tendency for students to feel like anonymous spectators rather than active, collaborative participants

- that is, there’s a real risk that many students will not properly engage with classroom activities, & that their learning will suffer as a result. I’ve written previously about flipped teaching as an example of a technique to increase student engagement (& performance), but with a range of different learning styles among class members, what works for one student won’t necessarily work for another.

So, how do Dougherty & Andercheck use social media to enhance their students’ engagement with the subject, and their achievement (as measured against the learning objectives for the paper)?

The larger a class gets, the harder it can be – even with the best will in the world – get everyone actively involved in discussions, debates and group work during class time. Teachers might try & manage this using a Student Learning Management System (SLMS) like Moodle but again, many students don’t really engage here either. (Certainly that’s been my own experience.)

The authors wondered, what about Facebook? After all,

[s]ocial media, such as Facebook, Twitter, and Instagram, are part of life for the generation of students now filling college classes

and it’s easy to load material and set up discussion threads. (Even a relatively technological illiterate like me can do it!) Why not use it as a more engaging SLMS, one that’s more likely to get buy-in from students because it’s already familiar to them?

I can just hear the cries of horror that might greet such a proposition. Don’t students already spend far too much time on FB and other networking sites? It would just be a distraction. These are valid objections. But with evidence in favour from a developing body of research into such uses of social media, Dougherty & Andercheck set up a study of the impact of a group FB page on students’ engagement & performance in their own class.

For anyone interested in doing likewise, their paper in Teaching Sociology has a very useful description of how the class page is set up & administered. (One of my Academy colleagues has similar pages for MOOCs that he is involved in; due to their size, he has some students help with the admin.) It was run in parallel with their ‘normal’ SLMS, Blackboard, and the latter was where students obtained class handouts & readings. FB was for sharing & discussion; for videos, news stories, & photos; for the ‘Question of the Day’.

For students unable to participate or uncomfortable participating in the classroom discussion, we invited them to add their thoughts and reflections to the conversation on Facebook. We used poll-style questions on the Facebook Group as another means to engage students.

Students readily got involved, ‘liking’ posts, joining discussions, and posting material. Two weeks into the semester, more than half the class had joined the page, and 2/3 were part of it by the end of the paper. To see how all this activity affected learning outcomes, the researchers carried out content analysis of student postings & matched this to performance, and also asked students for feedback via the usual paper appraisals.

The appraisal data showed that half the class visited the FB page on at least a weekly basis, and that the majority were positive about its effect on their experience in the class. While  24% disagreed (ranging from slight to strong disagreement) that it enhanced their experience, Dougherty & Andercheck noted wryly that “it was students who never or rarely used the Facebook Group who disagreed”. Students also felt that the page gave them a stronger sense of belonging in the course, and also that it positively influenced their achievement of the learning objectives.

Of course, the final proof of the pudding is in the eating (sorry, channeling cooking blog here!): was this reflected in actual performance? The researchers found that FB group membership showed a positive correlation to total quiz points and total points. It also had “a marginally significant, positive relationship” with both a student’s total score for the paper and their score in the final exam, and the number of posts someone made was linked to their quiz score.

What’s more, their analysis of the page’s content and their students’ use of the page clearly shows how involved many class members became in discussion. This is a big point for me: I use Moodle in my own class & it’s sometimes a bit sad to see how little real conversation there is about a topic. We might see a question posted, followed by a couple of answers, & then it all dies down again. Would discussions become deeper & more complex in a different, more familiar (&, let’s face it, less clunky) medium? I guess there really is only one way to find out. (And I’ll be making good use of the very helpful hints provided at the end of this thoughtful, and thought-inspiring, paper!)

K.D.Dougherty & B.Andercheck (2014) Using Facebook to Engage Learners in a Large Introductory Course. Teaching Sociology 42(2): 95-104 DOI: 10.1177/0092055X14521022

March 31, 2014

paying it forward

Over the last few weeks I’ve been mentoring a colleague from another institution, helping put together their portfolio for the 2014 Tertiary Teaching Excellence Awards nominations. It’s been a huge amount of work for them, given the need to encapsulate how they meet the award criteria in a total of 8000 words.

At first this looks an unreachable target, but then once you start writing notes and accumulating statements in support, then the problem becomes how to cut the thing down to size. And many people also find it really hard to write about themselves: it sounds like blowing your own trumpet & that can be a difficult thing to do. (Having said that, I know I looked my own finished portfolio & thought, wow! do I really do all that? It was quite affirming, plus the constant reflection was great for my teaching practice.)

So, it was a lot of work for my colleague, who wrote and edited many drafts, solicited supporting comments from students and colleagues, decided on a ‘theme’ to tie it all together, found suitable images – and all the while also carried their usual demanding teaching & admin roles. (I suspect the research may have taken a back seat for a while.) The end result: fascinating reading on a number of levels and a record of excellent teaching in practice (regardless of what happens in the TTEA stakes).

And on the other end of email & phone, I read those drafts, offered other possibilities for investigation/inclusion, proposed many edits (both large & small), found the occasional image, and suggested cuts – you reach a point where you’ve so much personal investment in what you’ve written that you just can’t bear the prospect of removing anything, no matter how the word limit looms over you**.

Yes, that took quite a bit of time at my end too, & I’ve had other colleagues at my institution asking why on earth I would want to take on such a task. But you see, I believe in paying forward: having won one of these awards myself, I feel that I should share what I’ve learned from the process and to help others with tasks like this.

And I’ve made a new friend as well!

 

**(I gather I also provided a calming influence :)  It’s been a great learning experience for me too, as I’ve learned about the cool things someone else is doing to enhance their teaching & their students’ learning experiences.)

February 11, 2014

musings on moocs

I’ve had a few conversations lately around the topic of Massive Open On-line Courses (or MOOCs). These fully on-line courses, which typically have very high enrolments, have become widely available from overseas providers (my own institution recently developed and ran the first such course in New Zealand, which I see is available again this year). If I had time I’d probably do the occasional one for interest (this one on epigenetics caught my eye).

Sometimes the conversations include the question of whether, and how much, MOOCs might contribute to what’s generally known as the ‘universities of the future’. This has always puzzled me a bit, as in their current incarnation most MOOCs don’t carry credit (there are exceptions), so don’t contribute to an actual degree program; they would seem to work better as ‘tasters’ – a means for people to see what a university might have to offer. Depending on their quality, they could also work to encourage young people into becoming more independent learners, regardless of whether they went on to a university – there’s an interesting essay on this issue here. So I thought it would be interesting to look a bit more closely.

Despite the fact that these courses haven’t been around all that long, there’s already quite a bit published about them, including a systematic review of the literature covering the period 2008-2012 by Liyanagunawardena, Adams, & Williams (2013), and a rather entertaining and somewhat sceptical 2013 presentation by Sir John Daniel, (based largely on this 2012 paper).

The term MOOC has only been in use since 2008, when it was first coined for a course offered by the University of Manitoba, Canada (Lianagunawardena et al, 2013). Daniel comments that the philosophy behind early courses like this was one of ‘connectiveness’, such that resources were freely available to anyone, with learning shared by all those in the course. This was underpinned by the use of RSS feeds, Moodle discussions, blogs, Second Life, & on-line meetings. He characterises ‘modern’ MOOCs as bearing little relation, in their educational philosophy, to these early programs, viewing programs offered by major US universities as

basically learning resources with some computerised feedback. In terms of pedagogy their quality varies widely, from very poor to OK.

Part of the problem here lies with the extremely large enrolments in today’s MOOCs, whereas those early courses were small enough that some semi-individualised interactions between students and educators were possible. Unfortunately the combination of variable pedagogy plus little in the way of real interpersonal interactions in these huge classes also sees them with very high drop-out rates: Liyanawardena and her colleagues note that the average completion rate is less than 10% of those beginning a course, with the highest being 19.2% for a Coursera offering.

Daniel offers some good advice to those considering setting up MOOCs of their own, given that currently – in his estimation – there are as yet no good business models available for these courses. Firstly: don’t rush into it just because others are. Secondly,

have a university-wide discussion on why you might offer a MOOC or MOOCs and use it to develop a MOOC strategy. The discussion should involve all staff members who might be involved in or affected by the offering of a MOOC.

His third point: ensure that any MOOC initiatives are fully integrated into your University’s strategy for online learning (my emphasis). To me this is an absolute imperative – sort the on-line learning strategy first, & then consider how MOOCs might contribute to this. (Having said that, I notice that the 2014 NMC Horizon report on higher education, by Johnson et al.,  sees these massive open on-line courses as in competition with the universities, rather than complementary to their on-campus and on-line for-credit offerings. And many thanks to Michael Edmonds for the heads-up on this paper.)

This is in fact true for anything to do with moving into the ‘universities of the future space (with or without MOOCs). Any strategy for online learning must surely consider resourcing: provision not only of the hardware, software, and facilities needed to properly deliver a ‘blended’ curriculum that may combine both face-to-face and on-line delivery, but also of the professional development needed to ensure that educators have the pedagogical knowledge and skills to deliver excellent learning experiences and outcomes in what for most of us is a novel environment. For there’s far more to offering a good on-line program than simply putting the usual materials up on a web page. A good blended learning (hybrid) system must be flexible, for example; it must suit

the interests and desires of students, who are able to choose how they attend lecture – from the comfort of their home, or face-to-face with their teachers. Additionally, … students [feel] the instructional technology [makes] the subject more interesting, and increase[s] their understanding, as well as encourag[ing] their participation… (Johnson et al., 2014).

This is something that is more likely to encourage the sort of critical thinking and deep learning approaches that we would all like to see in our students.

Furthermore, as part of that hybridisation, social media are increasingly likely to be used in learning experiences as well as for the more established patterns of social communication and entertainment (eg Twitter as a micro-blogging tool: Liyanagunawardena et al., 2013). In fact, ‘external’ communications (ie outside of learning management systems such as Moodle) are likely to become more significant as a means of supporting learner groups in this new environment – this is something I’m already seeing with the use of Facebook for class discussions and sharing of ideas and resources. Of course, this also places demands on educators:

Understanding how social media can be leveraged for social learning is a key skill for teachers, and teacher training programs are increasingly being expected to include this skill. Understanding how social media can be leveraged for social learning is a key skill for teachers, and teacher training programs are increasingly being expected to include this skill. (Johnson et al., 2014).

There is also a need, in any blended learning system, to ensure skilled moderation of forums and other forms of on-line engagement, along with policies to ensure privacy is maintained and bullying and other forms of unacceptable behaviour are avoided or nipped in the bud (Liyanawardena et al. 2013; Johnson et al., 2014). And of course there’s the issue of flipped classrooms, something that the use of these technologies really encourages but which very few teaching staff have any experience of.

Another issue examined by Liyanagunawardena and her colleagues, in their review of the MOOC literature, is that of digital ‘natives': are our students really able to use new learning technologies in the ways that we fondly imagine they can? This is a question that applies just as well to the hybrid learning model of ‘universities of the future’. In one recent study cited by the team, researchers found that of all the active participants in a particular MOOC, only one had never been involved in other such courses. This begs the question of “whether a learner has to learn how to learn” in the digital, on-line environment. (Certainly, I’ve found I need to show students how to download podcasts of lectures, something I’d naively believed that they would know how to do better than I!) In other words, any planning for blended delivery must allow for helping learners, as well as teachers, to become fluent in the new technologies on offer.

We live in interesting times.

And I would love to hear from any readers who have experience in this sort of learning environment.

T.R.Liyanagunawardena, A.A.Adams & S.A.Williams (2013) MOOCs: a systematic study of the published literature 2008-2012. The International Review of Research in Open and Distance Learning 14(3): 202-227

L.Johnson, S.Adams Becker, V.Estrada, & A.Freeman (2014) NMC HOrizon Report: 2014 Higher Education Edition. Austin, Texas. The New Media Consortium. ISBN 978-0-9897335-5-7

December 12, 2013

Evaluating teaching the hard-nosed numbers way

[This is a copy of a post on my blog PhysicsStop, sci.waikato.ac.nz/physicsstop, 10 December 2013]

Recently there’s been a bit of discussion in our Faculty on how to get a reliable evaluation of people’s teaching. The traditional approach is with the appraisal. At the end of each paper the students get to answer various questions on the teacher’s performance on a five-point Likert Scale (i.e. ‘Always’, ‘Usually’, ‘Sometimes’, ‘Seldom’, ‘Never’.)  For example: “The teacher made it clear what they expected of me.” The response ‘Always’ is given a score of 1, ‘Usually’ is given 2, down to ‘Never’ which is given a score of 5. An averaged response of the questions across students gives some measure of teaching success – ranging in theory from 1.0 (perfect) through to 5.0 (which we really, really don’t want to see happening).

We’ve also got a general question – “Overall, this teacher was effective”. This is also given a score on the same scale.

A question that’s been raised is: Does the “Overall, this teacher was effective” score correlate well with the average of the others?

I’ve been teaching for several years now, and have a whole heap of data to draw from. So, I’ve been analyzing it (for 2008 onwards), and, in the interests of transparency, I’m happy for people to see it.  For myself, the question of “does a single ‘overall’ question get a similar mark to the averaged response of the other questions?” is a clear yes. The graph below shows the two scores plotted against each other, for different papers that I have taught. For some papers I’ve had a perfect score – 1.0 by every student for every question. For a couple scores have been dismall (above 2 on average):

Capture1.JPG

What does this mean? That’s a good question. Maybe it’s simply that a single question is as good as a multitude of questions if all we are going to do is to take the average of something. More interesting is to look at each question in turn. The questions start with “the teacher…” and then carry on as in the chart below, which shows the responses I’ve had averaged over papers and years.
Capture2.JPG
Remember, low scores are good. And what does this tell me? Probably not much that I don’t already know. For example, anecdotally at any rate, the question “The teacher gave me helpful feedback” is a question for which many lecturers get their poorest scores (highest numbers). This may well be because students don’t realize they are getting feedback. I have colleagues who, when they give oral feedback, will prefix what they say with “I am now giving you feedback on how you have done” so that it’s recognized for what it is.
So, another question. How much have I improved in recent years? Surely I am a better teacher now than what I was in 2008. I really believe that I am. So my scores should be heading towards 1.  Well, um, maybe not. Here they are. There are two lines – the blue line is the response to the question ‘Overall, this teacher was effective’, averaged over all the papers I took in a given year; the red line is the average of the other questions, averaged over all the papers. The red line closely tracks the blue – this shows the same effect as seen on the first graph. The two correlate well.
Capture3.JPG
So what’s happening. I did something well around 2010 but since then it’s gone backwards (with a bit of a gain this year – though not all of this year’s data has been returned to me yet). There are a couple of comments to make. In 2010 I started on a Post Graduate Certificate of Tertiary Teaching. I put a lot of effort into this. There were a couple of major tasks that I did that were targeted at implementing and assessing a teaching intervention to improve student performance. I finished the PGCert in 2011. That seems to have helped with my scores, in 2010 at least. A quick peruse of my CV, however, will tell you that this came at the expense of research outputs. Not a lot of research was going on in my office or lab during that time.  And what happened in 2012? I had a period of study leave (hooray for research outputs!) followed immediately by a period of parental leave. Unfortunately, I had the same amount of teaching to do and that got squashed into the rest of the year. Same amount of material, less time to do it, poorer student opinions. It seems a logical explanation anyway.
Does all this say anything about whether I am an effective teacher? Can one use a single number to describe it? These are questions that are being considered. Does my data help anyone to answer these questions? You decide.

December 9, 2013

shaking up the academy? or how the academy could shake up teaching

Last week I spent a couple of days down in Wellington, attending the annual symposium for the Ako Aotearoa Academy. The Academy’s made up of the winners of the national Tertiary Teaching Excellence Awards, so there are around 150 or so of us now. While only 35 members were able to make it to this year’s event (& the executive committee will survey everyone to see if there’s a better time – having said that, everyone seems so busy that there’s probably no date that would suit everyone!), we had a great line-up of speakers & everyone left feeling inspired & energised. I’ll blog about several of those presentations, but thought I would start with one by Peter Coolbear, who’s the director of our parent body, Ako Aotearoa.

Peter began by pointing out that the Academy is potentially very influential – after all, it’s made up of tertiary teachers recognised at the national level for the quality of their teaching, & who foster excellence in learning & teaching at their own institutions.  But he argued – & I agree with him – that there is room for us to become involved in the wider scene. Peter had a number of suggestions for us to consider.

First up, there’s a lot going on in the area of policy – are there areas where the Academy might be expected to have & express an opinion? For example

  • There’s the latest draft of the Tertiary Education Strategy (TES), which “sets out the Government’s long-term strategic direction for tertiary education; and its current and medium-term priorities for tertiary education.” There’s a link to the Minister’s speech announcing the launch of the draft strategy here.
  • In addition, the State Services Commission’s document Better Public Services: results for New Zealanders sets out 10 targets across 5 areas. Targets 5 & 6 are relevant here as they are a reference point for government officials looking at evidence for success in the education sector. (Such scrutiny is likely to become more intense in light of the 2012 PISA results, which have just been made public.) Target 5 expects that we’ll “[increase] the proportion of 18-year-olds with NCEA level 2 or equivalent qualification”; #6 is looking for an increase in ” the proportion of 25 to 34-year-olds with advanced trade qualifications, diplomas and degrees (at level 4 or above)”. This will increase the pressure on institutions to increase retention & completion rates – might this have an effect on standards?
  • There’s also the requirement to achieve parity of success for ‘priority’ learners, especially Maori & Pasifika – this is priority #3 in the TES. (Kelly Pender, from Bay of Plenty Polytech, gave an inspirational presentation on how he weaves kaupapa Maori into pretty much everything he does in his classroom, in an earlier session.) And it’s an important one for us to consider. Peter cited data from the Ministry of Education’s website, ‘Education Counts’, which showed significantly lower completion rates for Maori & Pasifika students in their first degrees compared to European students, and commented that this will likely become a major issue for the universities in the near future.
  • If we’re to meet those achievement requirements, then how institutions scaffold learners into higher-level study, through foundation & transition programs, will become increasingly important. What are the best ways to achieve this?
  • Peter predicted increased accountability for the university sector (including governance reform). Cycle 5 of NZ’s Academic Audits has begun, and “is to be framed around academic activities related to teaching and learning and student support.” This is definitely one I’d expect Academy members to have an opinion on!
  • He also expects strengthened quality assurance processes throughout the education sector: this suggests a stronger (& more consistent) role for the  NZ Qualifications Authority, with the development of partnership dialogues across the sector (ie including universities).

Then, at the level of the providers (ie the educational institutions themselves – & that’s not just the polytechs & universities), we have:

  • a targeted review of qualifications offered at pre-degree level – there’s background information here;
  • a government-led drive to get more learners into the ‘STEM’ subjects (science, technology, engineering, & maths) – this poses some interesting challenges as, at university level, we’re seeing quite a few students who’ve not taken the right mix of subjects, at the right NCEA level, to go directly into some of the STEM papers they need for, say, an engineering degree;
  • the rise in Massive Open On-line Courses, or MOOCs. (I find these quite strange creatures as they are free to the student and typically attract very large enrolments, but also apparently have very low completion rates. What’s in them for the institution? A good way of offering ‘taster’ courses that hook students in?)
  • the likelihood that we will see the development of a system for professional accreditation of tertiary teachers (I’ve written about this previously and will write another post fairly soon, as accreditation was the subject of a thought-provoking session at the symposium);
  • how we achieve protection of academic standards – it’s possible that government policies (eg those linking funding to completion & retention rates) may result in a tendency to exclude of underprepared kids &/or lowering standards – neither is desirable but both are possible results of those policies.

That’s a big list and the Academy can’t do everything! So, what should it focus on? (This is not a rhetorical question – it would be great to get some discussion going.) The Academy, in the person of its members, is effectively a resource; a body of expertise – can it become a ‘go-to’ body for advice? Speaking personally I think we need to make that shift; otherwise we remain invisible outside our individual institutions & the teaching-focused activities we’re involved in, & in a politicised world that’s not a comfortable thing to be. Can we, for example, better promote the significance of teaching excellence outside the education sector? Become involved in the discussions around & development of any accreditation scheme? Develop position papers around maintaining teaching excellence in the context of the new TES?

What do you think? And what shall we, collectively, do about it?

November 21, 2013

learning leadership

and yes, that’s an intentionally ambiguous title :) (The full version was Learning leadership: the interplay between our own professional development and our classroom practice.)

I recently gave a pecha kucha** presentation on this subject, at an Ako Aotearoa mini-symposium up in Auckland. The idea for the subject of my presentation leapt to the front of my mind while I was at a Teaching Network*** meeting looking at how to raise the profile of teaching in tertiary institutions (specifically, universities). One of my colleagues kicked off part of the discussion with a brief talk on developing leadership in teaching, & I thought, all this applies to leading/guiding our students to become better learners, as well. Which is pretty much the thrust of my presentation. I pretty much use slides as talking points:

Slide2

 

Slide3

Slide4

Slide5

Slide6

Slide7

Slide8

Slide9

Slide10

And we finished up with some ideas on what future-focused leadership in teaching and learning could look like.

It would be really good to hear your thoughts on this :)

 

 

** a maximum of 20 slides, with a maximum of 20 seconds per slide. Certainly forces you to focus your ideas. Mine wasn’t that long, because I wanted to use it to spur discussion & so we need time for that in my short presentation slot.

*** an in-house group for staff from across the institution with an interest in all things to do with teaching.

August 23, 2013

what am i?

I’ve been involved in a few discussions lately, on the issue of what ‘we’ actually are. That is, are those of us who work with students in our lecture rooms, laboratories and tut classes, teachers? Is that the label we want attached to ourselves (eg in things like paper & teaching appraisal surveys)?

Disappointingly, there seems to be a fairly large body of opinion that says “no, no that’s not the right name. ‘Teachers’ is what people in schools could be described as. But we’re lecturers, not teachers.” (Someone went so far as to say that using the name ‘teacher’ would only be confusing, as students associated the term with their school experiences & didn’t expect it at university.)

Interestingly, this is not a reflection of how universities are described in the 1989 Education Act. Section 162 of the Act tells us (my emphasis in bold font) that

 universities have all the following characteristics and other tertiary institutions have 1 or more of those characteristics:

  • (i)they are primarily concerned with more advanced learning, the principal aim being to develop intellectual independence:

  • (ii)their research and teaching are closely interdependent and most of their teaching is done by people who are active in advancing knowledge:

  • (iii)they meet international standards of research and teaching:

  • (iv)they are a repository of knowledge and expertise:

  • (v)they accept a role as critic and conscience of society;

and that

  • a university is characterised by a wide diversity of teaching and research, especially at a higher level, that maintains, advances, disseminates, and assists the application of, knowledge, develops intellectual independence, and promotes community learning:

This all makes it fairly clear that the official view of what folks like me do, in our university jobs, is teaching i.e. facilitating advanced learning in our students, helping them to become independent, autonomous learners, and (while last, definitely not least!) promoting learning in the wider community. (I have to say, in Hamilton at the moment, this often feels like an uphill battle in the face of widespread misinformation about water fluoridation. But you can read more about this here, and here.)

And that’s true whether our job descriptions include the word tutor, lecturer, or professor. To me, if the word ‘teaching’ is included in the description of what universities do, then we are ‘teachers’.

Now, I suppose you could argue that I’m just being picky, but I think this is actually quite an important issue as it relates to what we perceive ourselves doing in our classrooms. That’s because if someone sees themselves as a lecturer, & not a teacher, then they could well have a mental image of what the role of ‘lecturer’ entails. And it’s a fair bet that this includes, you know, lecturing: standing in front of a class and delivering 50 minutes of information on a topic in which that person has expertise.

And to me, this is a problem because there’s an increasing body of research now that clearly shows that this passive-student model of teaching & learning – not just lectures, but also ‘cookbook’ lab classes – is probably the least effective thing we can do, in expanding students’ knowledge & understanding of a subject. This was demonstrated very clearly by Richard Hake in his 1998 analysis of the outcomes for more than 6,500 students enrolled in a total of 62 introductory physics courses. Hake found that courses that used ‘interactive-engagement’ techniques for teaching and learning were significantly better – much better – in terms of successful learning and retention of material. Subsequently Carl Wieman and his science-education research group have built on the work of Hake and others in the physics area – have a look at the figures at the end of this 2005 paper, for example: teaching techniques that encourage passive learning by students don’t result in any real long-term learning or retention. Nor is it just physics; I’ve written previously about similar research findings from the area of biology education (e.g. Haak et al. ,2011).

‘Teacher’ to me implies the use of a much wider range of classroom techniques that encourage active student engagement and successful long-term learning. And yes, I’m a teacher, and proud of it!

 

Haak DC, HilleRisLambers J, Pitre E, & Freeman S (2011). Increased structure and active learning reduce the achievement gap in introductory biology. Science , 332 (6034), 1213-6 PMID: 21636776

Hake RR (1998) Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. Am. J. Phys. 66: 64-74

Wieman C & Perkins K (2006) Transforming physics education. Physics Today Online, http://www.physicstoday.org/vol-58/iss-11/p36.shtml

May 2, 2013

science challenges & science education

The National Science Challenges have been announced – and have already received a lot of attention (including on Sciblogs, with posts by my colleagues GrantSiouxsie, and John - who also points at where the money’s going). What I’d like to address here is the comment by the Panel that it

was concerned by the lack of significant proposals in educational research

I have to admit that my first response to that was, well d’oh! Because, well, the public discussion was around national science challenges, I suspect that for many (most?) submitters the focus was to come up with a science-based proposal. After all (& please note bulging cheek ensconcing my tongue at this point), isn’t science education something that schools & other seats of learning ‘do’, rather than requiring science research? Hopefully not many scientists really think that way, & it’s great to see the additional Challenge, “Science & New Zealand Society” with its two goals (the first a science goal, while the second is societal):

To ensure the science capacities and literacy of New Zealand society so as to promote engagement between S[cience] & T[echnology] and New Zealand society, in turn enhancing the role played by science in advancing the national interest.

To allow New Zealand society to make best use of its human and technological capacities to address the risks and Challenges ahead. This requires the better use of scientific knowledge in policy formation at all levels of national and local government, in the private sector and in society as a whole.

 

Both are relevant to what follows here.

Let’s look more closely at the question of science literacy/appreciation/education for citizenship. The chair of the Panel, Sir Peter Gluckman, has previously made it clear that we need to do much more in engaging young people with science, to the extent of developing a science curriculum that focuses far more on science literacy than on accumulation of science knowledge. But what constitutes science literacy? This is something I’ve written about previously, & my fellow Scibloggers and I discussed it between ourselves more recently. So I was interested to find a set of nine science literacy ‘themes’ listed and expanded upon in a recent paper (Bartholomew & Osborne, 2004):

scientific methods and critical testing

science & certainty

diversity of scientific thinking

hypothesis and prediction

historical development of scientific knowledge

creativity

science and questioning

analysis and interpretation of data

cooperation and collaboration in the development of scientific knowledge

And while we might not agree on the relative order of these themes, or the completeness of the list, but they do give us something to go on with. (I’m going to talk about the formal education system for the moment – but I’m perfectly well aware that there’s much more than that to public engagement with science! Let’s just treat this as a starting point for discussion.)

Now, I’d like to think that the current NZ Science curriculum gives a good basis for developing these skills & attributes in all students Right Now, regardless of whether or not they intend to go on to study science at tertiary level. And let’s face it, most won’t, so we surely have to work on engagement with and understanding of what science is about, for all students. in fact, that’s a tension I struggle with myself: a proportion of my first-year biology students are taking the subject purely for interest, & in some cases haven’t studied the subject before. I want them to come away with an appreciation of the wonder and worth of the subject in their lives, as much as I want them to accumulate biological knowledge. It’s a tricky balancing act.

Anyway, while I might like to think that about the curriculum document, in reality I suspect that it doesn’t yet deliver. And that’s something that’s unpacked further by Bartholomew & Osborne, who note that there are a number of factors that affect teachers’ “ability to teach effectivelyabout science”.

One of those factors is the teachers’ own understanding of what science is all about, as opposed to their body of content knowledge. NB Please note, at this point, that this is not a criticism of teachers and the demanding work that they do; it’s a question of whether the training and experiences we offer our teachers prepare them well for this particular aspect of teaching science.

The researchers found that a reasonable proportion of the teachers they worked with were not really confident in their own ability to teach lessons based on the ideas embedded in those themes. This was partly due to uncertainties about their own knowledge, and partly around feeling that they lacked the classroom skills to deliver such a program. Which, of course, raises issues around provision of professional development opportunities (with the associated resourcing).

Related to that is their own engagement with the subject. OK, if you’re teaching the subject as a specialist science teacher, I’m guessing that you took this role on because you enjoy the subject and want to share that. But if someone’s a primary school teacher with very limited exposure to science during their training, then the story might be very different.

And so that would be a fruitful area for research, in NZ (and at this point someone is probably going to tell me that they’re Already Doing It): what is the actual level of science literacy – using, for example, those 9 themes listed above – in NZ science teachers at all levels? And how does that translate into classroom practices? And – if the answer is, not as well as we’d like – what do we do about it?

Teachers’ ability to enhance learning about science (as opposed to of science) is also affected by factors outside their classrooms. For example, the pressure is on, at senior school level, to ensure students do as well as possible in national assessment – which, for all the changes associated with NCEA, remains largely content-based. And classroom time is limited, so it’s easy to see how there can be more focus on content & less on the other desirable attributes. As Bartholomew & Osborne comment,

developing a questioning and sceptical attitude to scientific knowledge claims in students might actually be disadvantageous.

Perhaps that also needs to change. [Pace, Schol Bio examiners!]

 

H.Bartholomew, & J.Osborne (2004) Teaching students “ideas about science”: five dimensions of effective practice. Science Education 88: 655-682 doi: 10.1002/sce.10135

October 13, 2012

why kids should grade teachers

Next week my first-year biology students will be doing an appraisal of this semester’s paper, & of those academic staff involved in teaching it. They’re asked about the perceived difficulty of the paper, the amount of work they’re expected to do for it, whether they’ve been intellectually stimulated, the amount of feedback they receive on their work, how approachable staff are, & much else besides. (The feedback one was always my worst scoring attribute – until I asked the students what they thought ‘feedback’ met. It turned out that they felt this described one-to-one verbal communication. We had a discussion about all the other ways in which staff can give feedback – & the scores went up.) The results are always extremely useful, as not only to we find out what’s working, but we also discover what’s not (or at least, what the students perceive as not working) & so may need further attention.

Anyway, my friend Annette has just drawn my attention to a lengthy post in The Atlantic, by Amanda Ridley. It made fascinating reading.

In towns around the country this past school year, a quarter-million students took a special survey designed to capture what they thought of their teachers and their classroom culture. Unlike the vast majority of surveys in human history, this one had been carefully field-tested. That research had shown something remarkable: if you asked kids the right questions, they could identify, with uncanny accuracy, their most – and least – effective teachers.

Ridley, reporting for the Atlantic, was able to follow a 4-month pilot project that was run in 6 schools in the District of Colombia. She notes that about half the states in the US use student test data to evaluate how teachers are doing.

Now, this approach is fraught with difficulty. It doesn’t tell you why children aren’t learning something, for example (or why they do, which is just as interesting). And it puts huge pressure on teachers to ‘teach to the test’ (although Ridley says that in fact “most [American] teachers still do not teach the subjects or grade levels covered by mandatory standardized tests”). It ignores the fact that student learning success can be influenced by a wide range of factors, some of which are outside the schools’ control. (And it makes me wonder how I’d have done, back when I was teaching a high school ‘home room’ class in Palmerston North. Those students made a fair bit of progress, and we all learned a lot, but they would likely not have done too well on a standardised test of academic learning, applied across the board in the way that National Standards are now.)

So, the survey. It grew out of a project on effective teaching funded by the Bill & Melinda Gates Foundation, which found that the top 5 questions – in terms of correlation with student learning – were

  1. Students in this class treat the teacher with respect.
  2. My classmates behave the way my teacher wants them to.
  3. Our class stays busy and doesn’t waste time.
  4. In this class, we learn a lot almost every day.
  5. In this class, we learn to correct our mistakes.

and the version used with high school students in the survey Ridley writes about contained 127 questions. That sounds an awful lot, to me, but apparently most kids soldiered on & answered them all. Nor did they simply choose the same answer for each & every question, or try to skew the results:

Students who don’t read the questions might give the same response to every item. But when Ferguson [one of the researchers] recently examined 199,000 surveys, he found that less than one-half of 1 percent of students did so in the first 10 questions. Kids, he believes, find the questions interesting, so they tend to pay attention. And the ‘right’ answer is not always apparent, so even kids who want to skew the results would not necessarily know how to do it.

OK – kids (asked the right questions) can indicate is a good, effective teacher. What use is made of these results, in the US? The researchers say that they shouldn’t be given too much weighting, in assessing teachers – 20-30% – & only after multiple runs through the instrument, though at present few schools actually use them that way. This is important – no appraisal system should rely on just one tool.

That’s only part of it, of course, because the results are sent through to teachers themselves, just as I get appraisal results back each semester. So the potential’s there for the survey results to provide the basis of considerable reflective learning, given the desire to do so, & time to do it in. Yet only 1/3 of teachers involved in this project even looked at them.

This is a problem in the NZ tertiary system too, & I know it’s something that staff in our own Teaching Development Unit grapple with. Is it the way the results are presented? Would it be useful to be given a summary with key findings highlighted? Do we need a guide in how to interpret them? Do people avoid possibly being upset by the personal comments that can creep into responses (something that can be avoided/minimised by explaining in advance the value of constructive criticism – and by being seen to pay attention to what students have to say)?

Overall, this is an interesting study & one whose results may well inform our own continuing debate on how best to identify excellent teaching practice. What we need to avoid is wholesale duplication and implementation in our own school system without first considering what such surveys can & can’t tell us, and how they may be incorporated as one part of a reliable, transparent system of professional development and goal-setting. And that, of course, is going to require discussion with and support from all parties concerned – not implementation from above.

August 8, 2012

more on accreditation

I spent some time recently in an interesting discussion around the question of whether tertiary teachers should be required to complete some form of national accreditation. Now, many – but by no means all! – institutions do already have something like this available for their staff, albeit that take-up is essentially voluntary. What would happen to these in-house programs, we wondered, in the event of such a national qualification becoming the norm? Would the individual organisations stop running their own systems? – a pity, in many ways, as these are likely tailored to the needs of their own staff and students. There’s also the issue of portability: whether the putative national qualification would be portable, between institutions and between countries. If this could be guaranteed, then why would teachers bother with the in-house model? This would be a negative result overall, as it would then remove any need for an individual institution to develop and maintain its own programs for its own staff.

We also wondered what form accreditation – accreditation, not a qualification – should take. Teaching excellence is not a static thing: the best teachers are always reviewing, reflecting on, revising and enhancing their practice. A qualification based on examinations are not going to adequately measure these attributes. Far better, we thought, to go with portfoliosmeasured by portfolio of work. This would be a living document as the individual’s practice should be constantly self-reviewed & enhanced, a process reflected in the portfolio.

Part of the discussion hinged on just how you define ‘excellence’. We were all Tertiary Teaching Excellence Award winners, so you’d think we’d know, wouldn’t you? But we’re all excellent at different things, so a definition proved hard to pin down. Can we define ‘excellence’ a la John Hattie’s work on secondary teaching? Possibly. Well, maybe not ‘define’, but we could certainly give examples of excellence from the portfolios of previous TTEA awardees.  could then act as basis of any form of professional development. In fact, you could argue that those awardees show something called ‘positive deviance‘ – and in this instance ‘deviance’ is something to aspire to!

So maybe accreditation would be based on a portfolio – a ‘living’ document – demonstrating someone’s ongoing professional & personal development, & built around a clearly explained concept of ‘excellence’ as it applies to facilitating students’ learning (& helping others to do the same)? Something to be think about, anyway.

Older Posts »

The WordPress Classic Theme. Create a free website or blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 159 other followers