Talking Teaching

June 9, 2014

carl wieman on active learning

Recently I wrote about a paper by Freeman et al: a meta-analysis looking at the impact of active learning on student success in maths, engineering, & the sciences (the ‘STEM’ subjects). In the same volume of PNAS is an accompanying commentary by Carl WiemanWieman is a physics Nobel Laureate who also leads a research group working on improving teaching & learning in maths, engineering, & the sciences (which has resulted in some interesting initiatives at other institutions). Commenting on Freeman’s results, he notes that

Freeman et al. argue that it is no longer appropriate to use lecture teaching as the comparison standard, and instead, research should compare different active learning methods, because there is such overwhelming evidence that the lecture is substantially less effective. This makes both ethical and scientific sense.

Wieman goes on to say

However, in undergraduate STEM education, we have the curious situation that, although more effective teaching methods have been overwhelmingly demonstrated, most STEM courses are still taught by lectures – the pedagogical equivalent of bloodletting. Should the goals of STEM education research be to find more effective ways for students to learn or to provide additional evidence to convince faculty and institutions to change how they are teaching?

Personally I’d go for the former; there’s a wealth of information out there now. What’s needed now is to somehow get more university STEM educators to engage with the scholarship of teaching & learning in their various disciplines. Now there’s a challenge!

C.E.Wieman (2014) Large-scale comparison of science teaching methods sends clear message. PNAS published ahead of print, May 22 2014. http://www.pnas.org/cgi/doi/10.1073/pnas.1407304111

June 1, 2014

“If you’re going to get lectured at, you might as well be at home in bunny slippers”

This is a post I first wrote for the Bioblog.

There’s an increasing body of literature demonstrating the benefits of active learning for tertiary students taking science subjects. This is a topic I’ve written about before, but I’m always interested in reading more on the subject. And let’s face it, the more evidence the better, when you’re wanting to get lecturers in the sciences engaged in discussion around different ways of teaching. As you’ll have gathered, I find a lot of new science & education material via alerts on Facebook, as well as through the more conventional journal feeds & email alerts, and so it was with this recent paper by Scott Freeman & colleagues, which looks at the effect of active learning on student performance in science, technology, engineering and maths (STEM) classes: I saw it first described in this post1 (whence also comes the quote I’ve used as my title).

The paper by Freeman et al (2014) is a meta-analysis of more than 200 studies of teaching methods used in STEM classes, which included “occasional group problem-solving, worksheets or tutorials completed during class, use of personal response systems with or without peer instruction, and studio or workshop course designs” (ibid.). The impact of the various methods on student learning was measured in two ways: by comparing scores on the same or similar examinations or concept inventories; and by looking at the percentage of students who failed a course.

What did their results show? FIrstly, that students’ mean scores in exams assessing work covered in active learning classes improved by around 6% over more traditional teaching-&-learning formats (& finding that matches those of earlier studies); and secondly, that students in those traditional classes “were 1.5 times more likely to fail”, compared to students given in-class opportunities for active learning (with a ‘raw failure’ rate averaging 33.8% in traditional lecturing classes and 21.8% in more active classes). These results held across all STEM subjects. The researchers also found that active-learning techniques had a stronger effect on concept inventories compared to formal exams (& here I’m wondering if that doesn’t reflect – at least in part – the nature of the exams themselves eg did they give opportunities to demonstrate deep learning?) Interestingly, while the positive impact of active learning was seen across all class sizes, it was more pronounced in classes of less than 50 students.

On the class size thing, I’m wondering if that might be because it’s easier to get everyone actively involved, in a smaller class? For example, I’ve got a colleague at another institution who runs a lot of his classes as ‘flipped’ sessions, and ensures that all students get the opportunity to present to the rest of the group – this is far easier to set up in a class of 50 than in a group with 200+ students in it. (I know! When I run ‘design-a-plant/animal’ sessions, there’s time for only a sub-set of student ‘teams’ to present their creatures to the rest of the class. Plus you really have to work at making sure you get around all teams to talk with them, answer questions, & so on, and so it’s perhaps more likely that someone can remain uninvolved.)

The research team concluded:

Finally, the data suggest that STEM instructors may begin to question the continued use of traditional lecturing in everyday practice, especially in light of recent work indicating that active learning confers disproportionate benefits for STEM students from disadvantaged backgrounds and for female students in male-dominated fields. Although traditional lecturing has dominated undergraduate instruction for most of a millenium and continues to have strong advocates, current evidence suggests that a constructivist “ask, don’t tell” approach may lead to strong increases in student performance, amplifying recent calls from policy-makers and researchers to support faculty who are transforming their STEM courses.

The ‘bunny slippers’ quote from the lead author comes from the post that originally caught my eye. And I suspect there may well be bunny slippers (or the equivalent) in evidence when my own students watch lecture recordings at home :) But this does raise a question around massive open on-line courses (MOOCs), which tend to have a very high ‘fail’ rate – how much of this might be attributed to the difficulty in ensuring opportunities for active learning in these ‘distance’ classes?

And of course, we aren’t really talking a simple dichotomy between ‘traditional’ lecture classes and classes with a very high component of active-learning opportunities – something the research team also note: some of the ‘non-traditional’ methods they surveyed had only a 10-15% ‘active’ component. This is something discussed at more length by Alex Smith in a post entitled “In Defence of the Lecture”. I have to say that his approach sounds very similar to mine, with its mix of socratic questioning, pop quizzes, group discussions, and – yes – sections of ‘lecture’. As Small says:

Not every lecture is a person spending an hour talking nonstop to deliver facts. A good lecture is engaging, it naturally invites discussion and dialogue, it operates at a level much higher than raw information delivery, it is a natural setting for the expert to act as a role model, and it can be integrated with more formal activities (e.g., clicker questions, small-group discussions, etc.).

Lecture should not be the sole means of instruction, and bad lectures are a plague demanding eradication, but we err when we too strenuously inveigh against the lecture.

I couldn’t agree more. And maybe that’s a message that’s being lost in the louder discussion around active learning, and which needs to be heard more widely.

1 The comments thread for this story is also worth reading.

S.Freeman, S.L.Eddy, M.McDonough, M.K.Smith,N.Okorofor, H.Jordt & M.P.Wenderoth  (2014) Active learning increases student performance in science, engineering, and mathematics. http://www.pnas.org/content/early/2014/05/08/1319030111

February 7, 2014

not science as I know it

This was first posted on my ‘other’ blog :)

By accident,  I came across the curriculum document for Accelerated Christian Education (ACE) which provides teaching & learning materials to parents who are homeschooling their children. New Zealand students who complete the program right  to year 13 gain university entrance.

Home Schooling NZ gives parents advice about the ACE program, but makes it clear that HSNZ does not work for Accelerated Christian Education or sell their teaching & assessment materials.  However, I was startled to see the following listed by HSNZ as one of the ‘distinctives’ [sic] of the ACE program:

Each student is taught from a biblical perspective developing critical thinking skills that will enable them to discern what is truly “…the good and acceptable and perfect will of God.” (Romans 12:2)

Having had a fair bit to do with the development of the Science section of the current national curriculum document, specifically, the Living World component, I was naturally interested in seeing how ACE handles a science curriculum. The answer is, poorly.

In fact, I feel that it’s most unfortunate that the ACE science program is officially recognised here, given statements such as this from Sir Peter Gluckman (the PM’s Chief Science Advisor) about the importance of science and science education. For example, from the curriculum overview material for grade 1 students we learn that students will

  • [pronounce and learn] new vocabulary words as they are defined and used in the text
  • [discover] God’s wisdom as he1 learns about God creating Earth…
  • [learn] about the design and care of the human eye and ear; high, low, soft and loud sounds.
  • [learn] about the importance of personal health – clean teeth and hands.
  • [gain] a respect for God as he learns about God’s wisdom, goodness, kindness, and that all things belong to God.
  • [read] stories and answer questions about God’s creation.
  • [continue] to build eye-hand coordination by drawing shapes, irregular shapes, and directional lines.

That’s it.

In contrast, the New Zealand Curriculum document has a number of subject-specific achievement aims for students at this level, in addition to those relating specifically to the nature of science. For example, students in their first year or two of primary school should

  • Learn about science as a knowledge system: the features of scientific knowledge and the processes by which it is developed; and learn about the ways in which the work of scientists interacts with society.
  • Appreciate that scientists ask questions about our world that lead to investigations and that open-mindedness is important because there may be more than one explanation.
  • Explore and act on issues and questions that link their science learning to their daily living.

Remember, that’s in addition to the achievement aims for biology (Living World), chemistry (Material World), earth sciences (Planet Earth & Beyond). and physics (Physical World).

And so it continues. I mean, how could this (from the ACE objectives for Grade 3) be construed as science by anyone assessing the document?

Studies Bible topics such as Jesus’ return; sin, death, and the curse; man’s freedom to choose to love and obey God.

Or this?

Discovers the Bible to be the final authority in scientific matters.

Science, it ain’t. It would appear that helping students to gain and enhance critical thinking skills isn’t on the curriculum either – after all, teaching students to look to authority for the answers runs completely counter to encouraging critical thinking and teaching students how to weigh up evidence.

While I haven’t read all the PACEs available for the curriculum, partly because I am not going to buy them in order to do so, I have read through the samples available on line. Among other things, the materials I viewed encouraged rote learning rather than deep, meaningful understanding of a subject – a long way indeed from current best-practice models of teaching & learning.

However, others have read ACE’s PACE documents, & have been extremely critical of them. The Times Education Supplement, for example, was startled to find that ACE materials available in 1995 contained the claim that the Loch Ness Monster has been reliably identified and seems to be a plesiosaur. (It seems this reference has since been removed from new textbooks published in Europe.)

The TES also addressed some rather trenchant comments to the UK educational body responsible for giving the ACE curriculum equivalent status to O and A level examinations. Perhaps the NZ equivalent of that body should give the ACE documents a closer second look.

 

1 No female pronouns used, that I could see. (No room for female scientists in this curriculum, either – students are introduced to ‘early men in science’.)

 

December 13, 2013

can we help students too much?

One of the (many) good things about writing exams at a national level was that it really taught me how to write a good question: one that lacks ambiguity, clearly identifies what’s being asked of the students, and gives them opportunities to demonstrate their knowledge and understanding. So I was a bit saddened when marking exams recently to find that one question, in particular, wasn’t eliciting what I’d hoped for.

You see, I’d written a question in two parts. Students were first asked to summarise their knowledge the process of gene expression, and then – using the phrase “in your answer you should also” – asked to explain how various events might introduce diversity into the genome. And for some reason many students went straight past the first part and answered the second (and generally did this very well, I might add).

It’s the first time I’ve seen this to any marked degree and it really bothered me, & I’ve talked about it at some length with a colleague. We’re beginning to wonder if maybe we give students too much information around the exams. Now that may sound wrong – you’d think that demystifying exams, in the sense of giving access to previous papers so students get an idea of what to expect, would be helpful. It also highlights key themes in the subject that we return to again & again. We don’t just leave it at this, mind you: we spend time in tuts discussing things like how to ‘unpackage’ questions and how to plan the answer before actually writing it. But we’re wondering if making previous papers readily available in the study guide is leading to students ‘picking questions’ – deciding what’s likely to be asked, on the basis of what’s been asked in the past, and then not just studying around that but perhaps writing and learning ‘model’ answers off by heart. And regurgitating that answer regardless of what the question actually requires of them.

And that sort of learning is not the sort of learning I want to encourage.

So we decided to do things a little differently with next year’s study guide & include just a single year’s exam paper in it. Students will still be able to access many more through the library’s database; they’ll just have to do a bit more work themselves. And thinking more on this as I’m writing, perhaps we should also devote a bit of tut time to explicitly recognising those themes & working with the students to build up a ‘big picture’ view of the main ideas associated with them, by way of encouraging that breadth of understanding – and emphasising that this approach should allow them to approach with confidence any question we might ask.

Your thoughts?

December 12, 2013

Evaluating teaching the hard-nosed numbers way

[This is a copy of a post on my blog PhysicsStop, sci.waikato.ac.nz/physicsstop, 10 December 2013]

Recently there’s been a bit of discussion in our Faculty on how to get a reliable evaluation of people’s teaching. The traditional approach is with the appraisal. At the end of each paper the students get to answer various questions on the teacher’s performance on a five-point Likert Scale (i.e. ‘Always’, ‘Usually’, ‘Sometimes’, ‘Seldom’, ‘Never’.)  For example: “The teacher made it clear what they expected of me.” The response ‘Always’ is given a score of 1, ‘Usually’ is given 2, down to ‘Never’ which is given a score of 5. An averaged response of the questions across students gives some measure of teaching success – ranging in theory from 1.0 (perfect) through to 5.0 (which we really, really don’t want to see happening).

We’ve also got a general question – “Overall, this teacher was effective”. This is also given a score on the same scale.

A question that’s been raised is: Does the “Overall, this teacher was effective” score correlate well with the average of the others?

I’ve been teaching for several years now, and have a whole heap of data to draw from. So, I’ve been analyzing it (for 2008 onwards), and, in the interests of transparency, I’m happy for people to see it.  For myself, the question of “does a single ‘overall’ question get a similar mark to the averaged response of the other questions?” is a clear yes. The graph below shows the two scores plotted against each other, for different papers that I have taught. For some papers I’ve had a perfect score – 1.0 by every student for every question. For a couple scores have been dismall (above 2 on average):

Capture1.JPG

What does this mean? That’s a good question. Maybe it’s simply that a single question is as good as a multitude of questions if all we are going to do is to take the average of something. More interesting is to look at each question in turn. The questions start with “the teacher…” and then carry on as in the chart below, which shows the responses I’ve had averaged over papers and years.
Capture2.JPG
Remember, low scores are good. And what does this tell me? Probably not much that I don’t already know. For example, anecdotally at any rate, the question “The teacher gave me helpful feedback” is a question for which many lecturers get their poorest scores (highest numbers). This may well be because students don’t realize they are getting feedback. I have colleagues who, when they give oral feedback, will prefix what they say with “I am now giving you feedback on how you have done” so that it’s recognized for what it is.
So, another question. How much have I improved in recent years? Surely I am a better teacher now than what I was in 2008. I really believe that I am. So my scores should be heading towards 1.  Well, um, maybe not. Here they are. There are two lines – the blue line is the response to the question ‘Overall, this teacher was effective’, averaged over all the papers I took in a given year; the red line is the average of the other questions, averaged over all the papers. The red line closely tracks the blue – this shows the same effect as seen on the first graph. The two correlate well.
Capture3.JPG
So what’s happening. I did something well around 2010 but since then it’s gone backwards (with a bit of a gain this year – though not all of this year’s data has been returned to me yet). There are a couple of comments to make. In 2010 I started on a Post Graduate Certificate of Tertiary Teaching. I put a lot of effort into this. There were a couple of major tasks that I did that were targeted at implementing and assessing a teaching intervention to improve student performance. I finished the PGCert in 2011. That seems to have helped with my scores, in 2010 at least. A quick peruse of my CV, however, will tell you that this came at the expense of research outputs. Not a lot of research was going on in my office or lab during that time.  And what happened in 2012? I had a period of study leave (hooray for research outputs!) followed immediately by a period of parental leave. Unfortunately, I had the same amount of teaching to do and that got squashed into the rest of the year. Same amount of material, less time to do it, poorer student opinions. It seems a logical explanation anyway.
Does all this say anything about whether I am an effective teacher? Can one use a single number to describe it? These are questions that are being considered. Does my data help anyone to answer these questions? You decide.

September 23, 2013

teach creationism, undermine science

This is something I originally wrote for my ‘other’ blog.

Every now & then I’ve had someone say to me that there’s no harm in children hearing about ‘other ways of knowing’ about the world during their time at school, so why am I worried about creationism being delivered in the classroom? 

Well, first up, my concerns – & those of most of my colleagues – centre less on whether teaching creationism/intelligent design is bringing religion into the science classroom1, & more on how well such teaching prepares students for understanding and participating in biology in the 21st century. For example, if a school can make statements like this:

It is important that children and adults are clear that there is one universal truth. There can only be one truthful explanation for origins that means that all other explanations are wrong. Truth is truth. Biblical truth, scientific truth, mathematical truth, and historical truth are in harmony2.

and go on to list the “commonly accepted science we believe in”, then their students are not gaining any real understanding of the nature of science. And the statements regarding the science curriculum that I’ve linked to above indicate that it’s not just biology with which the school community has an issue. Physics, geology, cosmology: all have significant sections listed under “commonly accepted ‘science’ we do not believe in”3. (Did you notice the quote marks around that second mention of science?)

Science isn’t a belief system, & while people are entitled to their own opinions they are not entitled to their own facts. Any school science curriculum that picks & chooses what is taught on the basis of belief is delivering (to quote my friend David Winter) “a pathetic caricature of actual science, … undermin[ing] science as a method for understanding the world and leav[ing] the kids that learned it very poorly prepared to do biology in the 21st century.” Or indeed, to engage with pretty much any science, in terms of understanding how science is done and its relevance to our daily lives. And if we’re not concerned about that lack of science literacy, well, we should be.

 

although I do think this is a problem too.

2 with the subtext that the first ‘truth’ takes precedence.

Taken to its extreme, the belief system promoted in teaching creationism as science can result in statements such as this:

We believe Earth and its ecosystems – created by God’s intelligent design and infinite power and sustained by His faithful providence – are robust, resilient, self-regulating, and self-correcting, admirably suited for human flourishing…

…We deny that Earth and its ecosystems are the fragile and unstable products of chance, and particularly that Earth’s climate system is vulnerable to dangerous alteration because of miniscule changes in atmospheric chemistry.

This does not look like a recipe for good environmental management to me.

 

The WordPress Classic Theme. Create a free website or blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 159 other followers