Tag Archives: evaluation

Read more..

Great Student Feedback

Another Subversive Comment

I know that this is a rare post for me.  This blog hasn’t been action-packed this year.  However, this experience motivated me.

At this time of year, many of us teachers, particularly those of us in Korea, are likely dreading end of semester student evaluations.  Not only are these painful due to their impact on our employment status, but they are often worthless in terms of beneficial feedback.  Student evaluations can be wonderful sources of feedback on our teaching methods, materials, and overall classroom environment.  However, most evaluations focus more on what the administration cares about: Did you teacher come on time, where the materials good, and so on.  These are so vague that they are unlikely to help any concerned teacher improve their practice.

I’ve taken a number of approaches to this problem over the years, most included direct questioning, additional surveys (Google Docs is great for this), and even anonymous notes slipped under my door.  I don’t know why it’s taken so long for me to come to my most recent approach.  

I teach quite a few writing classes, which are some of my most popular courses (not necessarily my favorite to teach).  I’ve been doing the same courses long enough that I have my approach down, but I’m always in need of feedback on the topics, classroom management (grouping strategies), and new technologies or tasks.  This semester I taught a couple courses that focus on preparing students to take the Korean English teacher exam.  The class focuses on writing 2-3 paragraph responses to prompts relating to SLA, TESOL, and classroom management. I had the class write their feedback as a final in-class, timed writing assignment.  I was amazed with the feedback.

Now, first let me say that I was nervous before reading each one.  It’s tough to get feedback.  I don’t think anyone likes negative feedback, but it was required for this assignment.  They had to write 2 paragraphs: what should stay the same in this class and what should change in this class.  I knew that I was going to get some good criticism….and I did.

I generally don’t take isolated criticism to heart unless it is something that I was worried about previously.  There were plenty of these isolated problems that I really didn’t give too much thought about: too many/much reflections for the portfolio, provide detailed outlines for your lectures (never really a lecture in actuality), and so forth.

However, when I start seeing the same ideas/categories repeated, I take note.  Some are things that students universally agreed on where the benefits of peer review (this is the first year I’ve seen this agreed on), the number of assignments (this has been the biggest complain in the past…this year I had more), the creation of portfolios, and my “teaching style” (of course, they did this before I had given them a grade, LOL).  Agreement on what should be changed was a little more elusive: student presentations (as content for papers) were seen as a waste of time that could be better spent discussing (I see this being a group prepared handout next year) and many people commented on a variety of ways that peer editing could be improved (checklists, modeling, grouping, etc…).

There were many more suggestions that spanned the two categories.  Some were more commonly positive like writing topics and the class website, while others were more commonly negative like pacing and Twitter.  The Twitter feedback killed me.

I personally love Twitter. I’ve been using it with my writing classes for about 3 years now.  It was really difficult at first because nobody had heard of it, much less used it.  These days, few are using it, but everyone has heard of it.  I get more people each semester who find the value of authentic written interactions in Twitter.  These people, however, don’t make up for the masses who would rather not be bothered using it.  At this point, I think I will take the hint and kill it for this course.  I have other writing courses that are more general in nature and it will likely remain a requirement in those courses.  There is more room for using Twitter than in this course.

This kills me not because I like it but rather because I feel that students are missing out a great opportunity.  I’ve always justified using Twitter as a way for students to experience daily writing (a sentence a day) and to practice expressing themselves concisely.  I still think there are good reasons to use Twitter with a writing class, particularly Korean writers of English. These novice writers are more likely to write confusing, 20 clause sentences.

Facing inconvenient facts is one reason why I love this process.  These are things that I would never change if I didn’t have such overwhelming feedback.  I love my students for being so honest (at least I’m assuming there is a lot of honesty in there) and I am proud of them for not only providing feedback but doing so with such diplomatic language.  The harsh, blunt language I see with many of my novice writers was magically replaced with much softer, modal-rich language.  I must be doing something right when they can tell me to go to hell and I appreciate the way they say it 🙂

Amazing presentations (w/audio) on language teaching by Sue Swift

Selected Presentations, with audio, by Sue Swift (on AuthorStream).  I found this collection really amazing.  Not merely for the number of presentations, but for the great quality.  I’m using the listening presentations with my teaching listening class this semester as an introduction to the topics of the course.

I’m going to guess that Ms. Swift is a fan of Field’s view of listening instruction (or vice-versa) given that they dove-tail so wonderfully.  I no longer use the Field text with my students (too difficult), but I’m going to use many of the concepts.

Below are just some of the over 100 presentations that I found interesting.

Teaching Listening

Academic Discourses

Teaching Writing

Evaluating Written Work

Discourse Analysis

Error Analysis

History of Language Teaching

No Grading, More Learning – I’m totally going to do this. Will it work in Korea?

No Grading, More Learning

May 3, 2010

When Duke University’s Cathy Davidson announced her grading plan for a seminar she would be offering this semester, she attracted attention nationwide. Some professors cheered, others tut-tutted, and others asked “Can she do that?”

Her plan? Turn over grading to the students in the course, and get out of the grading business herself.

Now that the course is finished, Davidson is giving an A+ to the concept. “It was spectacular, far exceeding my expectations,” she said. “It would take a lot to get me back to a conventional form of grading ever again.”

Davidson is becoming a scholar of grading. She’s been observing grading systems at other colleges and in elementary and secondary schools, and she’s immersed herself in the history of grading. (If you want to know who invented the multiple choice test, she’ll brief you on how Frederick J. Kelly did so at Emporia State University and how he later renounced his technique.)

But it was her own course this semester — called “Your Brain on the Internet” — that Davidson used to test her ideas. And she found that it inspired students to do more work, and more creative work than she sees in courses with traditional grading.

Her approach — first announced on her blog — works based on contracts and “crowdsourcing.” First she announced the standards — students had to do all of the work and attend class to earn an A. If they didn’t complete all the assignments, they could get a B or C or worse, based on how many they finished. Students signed a contract to agree to the terms. But students also determined if the assignments (in this case blog posts that were mini-essays on the week’s work) were in fact meeting standards. Each week, two students led a discussion in class on the week’s readings and ideas — and those students determined whether or not their fellow students had met the standards.

via insidehighered.com

I heard about this when they first posted about Davidson’s plans and I thought they were a good idea. Nothing revolutionary (though this author seems to think it is), but I like the approach that she took.

First, she establishes a general learning contract stating that students who fulfill all of the assignments get an A. Not submitting an assignment or not meeting the criteria, results in a lower grade (not set in stone).

Next, she assigned students into expert groups. Their groups are responsible for a topic, which is about a week long. The other students are then given assignments related to the content from that week (in this case, a blog posting). The expert group reads the blog posts and determines whether the student fulfilled the requirements or not.

I’ve done group grading before in which members assigned a percentage grade to their peers (I then averaged the grades and that’s what they got) as part of the total grade for the assignment. It was terrible. The biggest problem was the differences in groups. Some groups has much higher expectations than other groups. Even when performance was high, the grades were low. This was a serious point of contention among the students (as seen on end-of-course evaluations) and something I will never repeat.

Davidson’s approach does 2 things that make it much better: (1) they only judge whether the criteria were met or not (not giving a grade). A good rubric would make this a relatively easy task. (2) It focuses on frequent, smaller assignments rather than bigger projects.

I’m going to give this a try next semester, but I have to find the right class to do it with. I will teach a pedagogical English class that this might be perfect for (same class peer grading failed in last time). Smaller, bit-sized, weekly assignments graded by weekly “expert” groups.

A hole that I see in this is the way this compartmentalizes learning, though. Weekly topic groups don’t necessarily do well at synthesizing information from previous topics. These higher order assessments might be best implemented as midterm and final projects, larger assignments that require demonstration of knowledge gained throughout the semester.

In this case, there would be two parts, student-graded work and teacher-graded work. The percentage of these two would be around 80% and could be split in half (maybe), with attendance/participation rounding out the last 20%.

This is going to require a little more thought, but I like where it’s going. I want students to take more responsibility for their learning AND the learning experience their classmates have. I feel that being engaged as both producers and critics will motivate them to go beyond the simple requirements of the course and will provide the course with more diverse perspectives on teaching and learning language in Korea.

Arizona Grades Teachers on Fluency

PHOENIX—As the academic year winds down, Creighton School Principal Rosemary Agneessens faces a wrenching decision: what to do with veteran teachers whom the state education department says don’t speak English well enough.

The Arizona Department of Education recently began telling school districts that teachers whose spoken English it deems to be heavily accented or ungrammatical must be removed from classes for students still learning English.

State education officials say the move is intended to ensure that students with limited English have teachers who speak the language flawlessly. But some school principals and administrators say the department is imposing arbitrary fluency standards that could undermine students by thinning the ranks of experienced educators.

via online.wsj.com

Really, they are not grading on fluency. At least in this article, fluency is not really addressed. They are really looking at some sort of target accent and grammar use measures. I’m really wondering if the WSJ just didn’t report this with enough accuracy. I can’t imagine that they (the Arizona DoE) would be that messy in proposing evaluation measures.

Honestly, I don’t know what to think of this. The racist scumbags are out in force if you take a look at the comments section, and this is enough to make anyone thing this is a bad idea. However, it is much more than an issue of race or even language identity. This is the ongoing, knock-down, drag out fight on the issue of NESTs (native English-speaking teachers) and NNESTs (non-native English-speaking teachers) in ESL classrooms.

This debate has been hot in TESOL for many years. The growth of interest/belief in World Englishes has kept it at the forefront of criticism, theory, and practice discussions in recent years.

Ordinarily, I fall on the side of the NNESTs on this argument, but my opinion differs depending on the context and the goals of the organization. The policy, at first glance seems reasonable. Teachers with an accent or grammar that impedes communication, should be removed from the classroom (Arizona is only proposing that they are removed from ESL classrooms). This is completely reasonable, BUT….

Oops, we now have the problem of rating these teachers. Should all teachers be accessed by this measure? That would only make sense. There are plenty of native English-speaking teachers out there with terrible grammar and writing skills (also referred to in the article). We should get rid of them to. Or, should the people evaluated just have to be as good as the worse of the native English-speaking teachers? That would set a low bar, wouldn’t it?

What about a teacher with a heavy Scottish accent? I mean, have you ever seen Trainspotting? It may be English, but it’s pretty tough to understand for most Americans. I’d even venture to guess that most Americans understand English with a heavily influenced Spanish accent better. Really, we hear it much more often. So, the Scottish are out. While we are at it, the English, Australians, New Zealanders, and South Africans should be out, too. If they don’t speak American they shouldn’t be teaching our fragile children. Oh wait. Canadians. They’re out too. What’s up with that “aboot” thing. That ain’t American, ya know? They’re gone.

OK, so I lapse into a good deal of sarcasm. The question is left unanswered, though. What is the target? This is the slipperiest of slopes in a country where there is no standard. No matter where you live, everyone will insist that their English is standard. That doesn’t mean it won’t impede comprehension when interacting with students from other regions. If we have this much variability at home, what is the standard that we shoot for?

I don’t outright disagree with the Arizona policy; however, I am doubtful that they can come up with a fair assessment of these abilities that take into account the many factors that make a good teacher of English.

%d bloggers like this: