Talking Teaching

March 14, 2012

how do you give feedback to university teachers

How do you give feedback to university teachers? – this was the search ‘topic’ used by one visitor to Talking Teaching. It struck a chord with me as I’m part of a small group of people discussing that very question, so I thought it might be a good topic for a blog. Not least because actually sitting down & writing about it should help to focus my own thoughts on the issue.

My institution expects teaching staff to carry out regular appraisals of their papers & their teaching in those papers.While there are a number of ways to do this, in practice most people use the ‘standard’ form: a set of Likert-scale questions on both paper & teachers that are common to all appraisals; a set of open-response questions (identify 3 things about this paper/teacher that should be changed/kept the same); &, if the lecturer chooses, some other questions as well. (Last year I included a set about student’s perceptions of Panopto, for a research project that I’m running with a couple of colleagues.) So there’s potentially quite a bit of information available there.

It’s what happens to this information, of course, that matters. Here, the current state of play sees lecturers receive a summary of the Likert question responses, plus any demographic information, fairly soon after the semester ends. Once the grades for the semester are finalised, we’re then sent the original survey forms, so we can then read the open-ended material as well. Both lots of information are potentially extremely useful if you’re wanting to improve paper delivery & your own teaching. The thing is – does everyone actually read it? Anecdotal evidence would suggest not: that the sheaf of paper may sometimes simply be flicked through (at best) before relegation to the paper-recycling device commonly known as a rubbish tin. When this happens, both students & teacher miss out. The students have spent time engaging with the questionnaire & do have a right to expect that their words will be read & (hopefully) responded to. And the lecturer may have missed out on suggestions that might allow them to enhance their paper’s delivery. And of course, there’s no closing of the feedback loop – letting the class know that you’ve read their comments & suggestions, & explaining how & why (or why not) you’re intending to respond to them. This in turn can see students becoming quite disillusioned with the whole process.

One of the options we’ve discussed, as a means of improving this part of the system, is whether to provide teaching staff with a summary of the open-ended questions as well, perhaps with a commentary alongside: “X% of the class felt that…. This suggests that… – have you considered the following.. ?” This, of course, would constitute a lot of extra work for our Teaching Development staff!

And there’s also the question of whether this is the best, or the only, way of getting feedback on one’s teaching.What about on-going formative feedback during the semester, using techniques like one-minute papers or ‘muddy questions’ (in which students highlight the points in a lecture that most puzzled or confused them)? Or the use of feedback surveys in learning management tools like Moodle? There’s also the issue of perceived legitimacy – I’ve heard it said that students don’t know enough about a given subject to give any meaningful comment. (While this is likely true about the content it’s certainly not the case for the methods – students do have a fairly good idea of the teaching styles & tools that work best to enhance their learning.) Would feedback be better coming from peers rather than students? How comfortable would lecturers be with having a colleague sitting in on their classes & providing constructive comments afterwards?

I seem to be posing more questions than I’ve answered! Please feel free to weigh in with your suggestions :-)

Advertisements

February 26, 2012

in the rush to ‘e-learning’, are we losing sight of our goals?

One of the ‘big things’ in schools these days seems to be the increasing expansion of e-learning. I’ve written previously on one school’s decision to require all its new students to have iPads, or similar tablet-style computers. At the time I worried about whether, in the rush to embrace new technology, the question of whether its use would enhance student learning was being left behind.  And a friend of mine who’s a secondary teacher recently said something similar: these technologies can be tools for learning but do not & should not replace the need for linking our teaching to a student-inquiry-based experiential and cognitive-conflict-based learning (which requires a lot of forethought & planning from teachers!).

That concern resurfaced yesterday as I was reading the NZ Herald‘s on-line edition (on my iPad, lol), & found one story citing a couple of US reports suggesting that perhaps e-learning isn’t all it’s cracked up to be.

The first of the Herald‘s references was to this report at Education News Colorado, which examines the performance of students who are taught entirely on-line (for a range of reasons, that could include having dropped out of  ‘regular’ schooling, living in an extremely isolated area, or for philosophical reasons. At this point I need to note that the news report is based on an analysis of on-line school data, & so far doesn’t appear to have been published in the science education literature. (However, the Colorado Department of Education annual report, from which the data are drawn, can be found here.) Nonetheless, the analysis does appear to highlight some rather worrying trends:

  • Online students are losing ground. Students who transfer to online programs from brick-and-mortar schools posted lower scores on annual state reading exams after entering their virtual classrooms.
  • Academic performance declined after students enrolled in online programs. Students who stayed in online programs long enough to take two years’ worth of state reading exams actually saw their test results decline over time.
  • Wide gaps persist. Double-digit gaps in achievement on state exams between online students and their peers in traditional schools persist in nearly every grade and subject – and they’re widest among more affluent students.

Now, one reason put forward by education officials for the apparently wide differences in results was that on-line education was pretty much an option of last resort, & certainly at least one Colorado virtual school does appear to target at-risk students who may well be behind on many educational indicators. However:

The analysis of state data shows, however, that most online school students do not appear to be at-risk students. Only about 120 students of the more than 10,000 entering online programs last year were identified as previous dropouts returning to school, and only 290 entered online schools after spending the prior year in an alternative school for troubled youth.

The obvious question is, why? Because there does appear to be something going on. And it’s relevant to NZ even though fully on-line teaching is a long way from the use of iPads & their like in a bricks-&-mortar classroom: we’re still looking at two stages on a continuum here.

Part of it could be that kids are not really as tech-savvy as we’d like to think. Putting them in front of a desktop computer, or giving access to things like tablets, doesn’t mean that they’ll necessarily use the technology to its best advantage. They may well need to learn that skill. And those using the technology to teach also need to think about how well it fits their learning objectives – is it there because it’s ‘there’, or because it enhances learning in some way.

Coming back to the full-blown exclusively on-line learning thing: there are also issues of community & pedagogy. In a real (as opposed to virtual) school, students are part of an actual community that includes both their peers & their teachers, & which can extend into the community outside of school. It can be rather isolating to be a distance student, & not be a part of that (this was certainly my experience when I was studying extramurally for my teaching qualification).

Which is where the pedagogy comes in. Certainly from a university perspective, we haven’t always been terribly successful at moving from the face-to-face to the on-line teaching environment. However, technologies like vide0-conferencing, skype, moodle & panopto can help to give some sense of belonging to a learning community – as can tailoring teaching materials to this alternative means of teaching & learning, instead of simply uploading everything in the format that’s used in ‘normal’ classes. Are some of the students in the Colorado study missing out on that sense of community?

And the Herald‘s second reference? It was to this story (from September 2011) in the New York Times, which carried out what looks like a fairly extensive investigation on the use of technology in schools, before concluding that

schools are spending billions on technology, even as they cut budgets and lay off teachers, with little proof that this approach is improving basic learning.

Now, that’s talking about the current status quo in parts of the US. New Zealand’s a long way back from what the NYT is describing, both in the extent of our technology roll-out & in the amount of money we have available for it.  And the research into the effectiveness of on-line teaching & learning is certainly being done (here, here, here & here, for example). (There’s also an interesting review of ‘virtual schools’ available here, which uses New Zealand as one of its examples.)

But still: technology, in education as elsewhere, is a useful tool, but not necessarily a panacea for all ills.

January 24, 2011

changing the culture of science education at research universities

This is a cross-post of something I’ve just written for my ‘other’ blog :)

 That’s the attention-grabbing title of a new paper in Science magazine’s ‘education forum’ section (Anderson et al. 2011). Most readers will know that science education is a subject dear to my heart, & a topic that Marcus & I write on from time to time (here & here, for example). The authors are all professors at the Howard Hughes Medical Institute & are supported by that institution to create ‘new programs that more effectively engage students in learning science’ (ibid), so I was keen to see what they had to say on the topic of raising the profile and status of teaching at the tertiary level.

In the opinion of Anderson & his colleagues (& it’s an opinion that I share)

Science education should not only provide broad content knowledge but also develop analytical thinking skills, offer understanding of the scientific research process, inspire curiosity, and be accessible to a diverse range of students.

 Now, you might think, ‘well, obviously!’, and certainly all my colleagues would agree that these are good aims, but the devil’s in the detail. All institutions have what are called ‘graduate profiles’, & ideally when new curricula are being developed, or existing ones reviewed, their relevance to that graduate profile should be at the forefront of everyone’s minds. The difficulty, though, is that most university lecturers aren’t trained teachers but have generally ‘picked it up on the job’. They’re not familiar with the science education literature &, with all the pressures on them to generate external funding and maximise their research profile, it’s going to be hard to take the time to find and read relevant material. Heck, at the moment I struggle to find time, and that’s in my research area!

Anderson et al argue that turning this around requires a culture shift at the level of the institutions themselves, suggesting that these institutions need to “more broadly and effectively recognise, reward, and support the efforts of researchers who are also excellent teachers.” They list 7 initiatives that would move things along towards this end.

Educate faculty about research in learning. There’s a wealth of literature out there on ways to enhance teaching and student learning. (I’m reading some of it myself at the moment.) But the key thing here is time. Without time for researchers in any given discipline to sit down & get a a feel for the education literature (without feeling guilty about not spending that time reading in their ‘own’ field, applying for research grants, supporting research students, or teaching…), & to play around with some of the ideas therein, this will be a long, slow process. Maybe a grassroots approach might be better, more engaging? At my institution we’ve got ‘teaching advocates’ (Marcus is one) who organise informal lunchtime sessions for people to sit down & discuss particular teaching approaches, or maybe just throw ideas around. These are good ways of getting discussions going & supporting people in what they’re doing in the classroom.

Create awards and named rofessorships that provide research support for outstanding teachers. Well, we certainly have awards: in-Faculty & cross-campus at this institution & all others I can think of, plus the national Ako Aotearoa awards. And it’s jolly nice to get one, too! But a question that I’d rather like to look into is, what is the wider impact of these awards? They’re nice for the awardee (in a time when the purse-strings are tight, it’s nice to know that you’ll be able to go to a couple of relevant, conferences without having to think too hard about how to fund it!), but do they change the attitudes & perceptions of others on-campus? Do they have a lasting impact on institutional culture?

Require excellence in teaching for promotion. The authors argue, & I agree, that this needs to be a broad-brush approach, not restricted to looking at data from end-semester course appraisals. They say, “[we] must identify the full range of teaching skills and strategies that might be used, describe best practices in the evaluation of teaching effectiveness (particularly approaches that encourage rather than stifle diversity), and define how these might be used and prioritised during the promotion process.” And as part of this we need to encourage people to try new things. There’s a real worry, & risk, that trying something new in the interests of improving your teaching will backfire: if for whatever reason the students don’t like what you’re doing, those end-semester scores may well decline as a result. Which is why these shouldn’t be the only way of measuring teaching quality and effectiveness. (This, of course, requires that the people involved in determing promotion rounds need to be aware of the existence & value of other means of assessing teaching quality.)

Create teaching discussion groups. the teaching advocate meetings run by Marcus & his counterparts, & the institution’s ‘teaching network’ meetings, are developing a nucleus of such groups. Maybe members of these groups might be interested in working on peer assessment of teaching? You can learn an awful lot from watching other experienced practitioners in action – I know I do. It can be a bit nerve-wracking, having another teacher sit in on your classes, but the discussions afterwards can be really rewarding. (In that regard, something like panopto is an excellent tool to aid reflection on your own teaching, if you’d rather someone else didn’t sit in & give you feedback.)

Create cross-disciplinary programs in college-level learning. Or maybe even just cross-disciplinary discussions. When I taught at high school, everyone was involved in staff meetings, so you had plenty of opportunity to talk with people teaching in other subjects. You tend to lose that sort of collegiality in large tertiary institutions, because every Faculty, & sometimes every department, will have its own tearooms & meeting spots. And that’s a pity, really, because unless you go out of your way to meet your counterparts in other parts of the organisation (or even just go to one of their in-house seminars), you can be closed off from some really interesting discussions about research & practice. (But yes, it is hard to find the time. Time, again; that really does seem central to all this.)

Provide ongoing support for effective science teaching. This can potentially be expensive up-front, but has long-term benefits in terms of student engagement & outcomes. Expensive, because students learn science best when they’re engaged in doing science – & this means lab & field work, as often as not.  But how else are students to learn what it is to ‘do’ science, & to become really engaged in that doing?

And finally, Anderson & his colleagues recomment engag[ing] chairs, deans, and presidents (in NZ, a ‘president’ would be a vice-chancellor), because institutional leadership is crucial in bringing about such changes. These leaders – & in fact, all involved in teaching & learning, need to

foster a culture in which teaching and research are no longer seen as being in competition, but as mutually beneficial activities that support two equally important enterprises, generation of new knowledge and education of our students.

Anderson WA, Banerjee U, Drennan CL, Elgin SC, Epstein IR, Handelsman J, Hatfull GF, Losick R, O’Dowd DK, Olivera BM, Strobel SA, Walker GC, & Warner IM (2011). Science education. Changing the culture of science education at research universities. Science (New York, N.Y.), 331 (6014), 152-3 PMID: 21233371

June 13, 2010

the tyranny of powerpoint

Filed under: education, science teaching — Tags: , , — alison @ 11:31 pm

This is a re-post of something I originally wrote for my ‘other’ blog, as a result of reading a thought-provoking paper about powerpoint that was given to me by a colleague.

I began my university teaching career in the years B.P. (Before Powerpoint). Blackboards, chalk, & overhead transparencies (often hand-written & hand-drawn) were the order of the day. Since then, Powerpoint has become an almost universal tool & ‘chalk-&-talk’ is a rarity. But Powerpoint is just a tool, & using it doesn’t guarantee a good presentation. (Slides that simply present large blocks of text; blocks of text in tiny fonts; lines of text that ‘fly’ in from one side or the other; typewriter sounds as letters appear on the screen – don’t do it! Please don’t go there!)

Anyway, a colleague has just given me a copy of Yiannis Gabriel’s 2008 paper looking at the use (& abuse) of Poweroint as a teaching tool. And it’s really got me thinking.

Gabriel begins by noting that Powerpoint “accomplish[ed] what earlier technologies did (overhead transparencies, slides, chalk and blackboard) only more efficiently, more stylishly.” However, it’s probably had more widespread, more pervasive effects: Powerpoint has become the basic lecture  tool, but simply relying on it without thinking about how it’s used can have some far-reaching effects on the nature of the learning that goes on in lecture theatres. One of his concerns is that, while Powerpoint is great for showing information in visual form (graphs, diagrams, photos, embedded videos), it may also affect students’ abilities to analyse & think critically about information. (It can also act as a prop – how many lecturers these days would feel comfortable giving a lecture without powerpoint, if the power goes down or the technology fails?) In fact, he expresses his own concern that “Powerpoint inevitably leads to comfortable, incontestable, uncritical, visually seductive and intellectually dulling communication.”

Now, like almost all my colleagues I use Powerpoint on pretty much an everyday basis, & so Gabriel’s ideas gave me considerable food for thought. It’s easy to slip into using this technology routinely, in a way that’s really just ‘chalk-&-talk elevated to another level. I try hard to avoid this: I use images & phrases as something to talk around & as cues for students to think about concepts, & I try to encourage discussion around the ‘big ideas’ of each lecture, using things like pop quizzes to start things off. (I really enjoy it when students ask probing questions that require a bit of thought for me to answer properly, not least because it lets me model how scientists think about things.) But is this enough?

Certainly the technology has its shortcomings, although these tend to be in how it’s applied rather than inherent in Powerpoint itself. You’ve planned your lecture in advance, all the images & words are assembled onto your slides – how easy is it to deviate from this if during the course of the lecture it becomes obvious that some in the class don’t understand what you’re saying, or want to ask questions around a particular issue? It could be argued that you just have to get through that material – it’s needed as the basis for the next lecture or some other paper – & the students will have to come to tutorials or ‘office hours’ to fill the gaps. But by then the moment’s passed.

Myself, I don’t see the value in that. Better by far to address the issues that students raise, on the spot – after all, how can I expect them to understand the material that follows if they haven’t ‘got’ what I’m talking about at the moment? You can deal with this with Powerpoint, as you would have done in the ‘old days’: I had the experience a few weeks ago where it became clear that many in the class hadn’t a clue about meiosis, & without it much of the rest of the lecture wasn’t going to make much sense to them. We ended up with an impromptu tutorial, with me using the computer mouse to ‘draw’ on my slides (having changed it from the usual arrow to a virtual felt-tip pen) to illustrate the points we were talking about. Yes, we didn’t get through everything I’d intended to for that class – but I was able to do an extra panopto recording later that day for the students to follow, & there were always the tutorials…

So I thought I was doing OK – & then Gabriel mentioned bullet-point lists… These are pretty much the standard way to present information in Powerpoint, but Gabriel points out that they contain some fish-hooks for teacher & student alike: “many people (and most  students) confronting a list will assume that it is exhaustive, that the items on it are co-equivalent…, and that they are mutually exclusive. In reality, few lists meet these requirements, and yet they block thinking into precise areas of overlap or items that are absent from the list.” There’s also a risk that students will see the lists as completely authoritative where they may actually be tentative. And it’s easy to use them to gloss over things that the lecturer’s not sure about, or doesn’t want to discuss – just don’t put those items on the list! 

When I think about it, I can see some of these things coming through in students’ test papers. For example, in teaching about the different ‘major phyla’ of animals, it’s easy to list the key features of each phylum in a series of bullet-points. I make the point in lectures that there may be other interesting features in a particular phylum – but in a test, for many students it’s as if I’d never said that; the bullet-point items seem to be all-important. This suggests to me that these students haven’t thought about other things that were said in lecture, or maybe those other things didn’t even register. And it’s made me wonder if there are other steps I could take to get this information across in a meaningful way that prompts the class to think carefully about what’s being said & why it matters.

Gabriel criticises images as well. And I agree with him – it’s quite easy to put together a sequence of images that can engross the audience, to the point where they don’t actually think critically about what’s being said. But I also strongly agree that it can enhance student learning & understanding of things like anatomy or physics. Diagrams, too, are a double-edged sword. Used simply to present large amounts of information they can be both boring & overwhelming – but they can “also open up new possibilities of creative thinking, communication and learning.”

I can see that I’ve got a lot of thinking and reorganising to do. I’d like to re-jig my Powerpoints to encourage a number of skills in my students, to enhance their learning – and because many of the skills that Gabriel identifies as desirable emphasise aspects of the nature of science itself:

  • filtering out the irrelevant & focusing on the memorable and significant;
  • tolerating uncertainty;
  • coping with ambiguity;
  • recognising & enjoying the fact that we don’t have clear, permanent solutions to every puzzle & problem;
  • developing the capacity for analytical, critical thought.

Using Powerpoint in a way that goes beyond it being merely a tool for presenting information can only enhance students’ learning (& – speaking personally – my enjoyment of teaching).

Y.Gabriel (2008) Against the tyranny of powerpoint: technology-in-use and technology abuse. Organisation Studies 29: 255-276. doi: 10.1177/0170840607079536. Document available online at http://oss.sagepub.com/cgi/content/abstract/29/2/255

March 5, 2010

the sound of my own voice

Filed under: science teaching — Tags: , , — alison @ 2:49 pm

Every now & then I take a timid step into the unknown. ‘Unknown’ for me, that is. Today, the step was to record my lectures using ‘coursecast’ (Panopto) software, so that my students can view them again (& again & again… heck, I hope not – if they need several viewings then my explanations etc probably aren’t up to scratch!)

I must say, I was quite pleased with the result. Students viewing a recording get to see a smallish ‘live action’ screen that shows me plus the backs of a few rows of heads; a list of slides that allows them to jump from one to the other; the current slide plus thumbnails of the others, which allow allows them to jump between slides; & if enabled, a view of the computer screen is also available. The quality of the recording is good – but I must say, it’s quite strange listening to your own voice! Sounds different when it’s actually echoing around in your own head.

I didn’t use the ‘screen capture’ option & now I think I should have done, & will in future. This is because I routinely ‘draw’ on my powerpoint slides – I always use the arrow (cursor) as a pointer, because it’s much easier on the eye than a jiggly laser pointer spot, but you also have the option of using it as a pen. This lets me cross things out (if I’ve made a typo, for example!), underline for emphasis, & draw scribble a diagram for explanation. I find it really helps the classroom dynamic as well – lets the students see I don’t mind who knows that I’ve made a mistake, plus it can inject a bit of humour. Anyway, those scribbles don’t show on the slide capture function, so I’ll enable ‘screen capture’ from now on.

Now, here’s the philosophical musings… It could be argued – & I suspect many of my colleagues will do this – that if my lectures are recorded & available after the event, that the students won’t bother coming at all. (They said this when I started putting all my ppts on moodle ahead of lectures, & I didn’t notice any obvious drop in numbers!) Personally, I don’t think it’s all that likely – there’s a lot of interplay in my lectures that the recording won’t pick up, because it’s student-based & I’m the only one wired for sound. And there are a lot of benefits to be had from doing this sort of thing. Students who are ill won’t have to rely on study guides or their friends’ notes but can still see the performance. And students who didn’t catch a comment, or who need to hear something again, can replay it. Similarly, if they didn’t understand the first time, they’ve got the opportunity to hear things again. It’s got to be good for students.

Good for me, too: I get to see (in miniature) & hear what the students see & hear, so if I’ve got any irritating mannerisms etc then I can identify them & – I hope! – work to correct things.

So hopefully it’s a win-win for everyone :-)

« Newer Posts

Blog at WordPress.com.