Student Peer Review Process? Here’s My Version

When I tweeted yesterday that I was creating a peer review process for my students to assess each other’s writing and provided a sample of what I was up to, I had many requests to share the whole process.

I am happy to do that and will do so here in this post with two caveats:

  1. This process is untested. This means, I can’t even give you anecdotal feedback as to if it works. So for now I will share the materials but it is probably wise to warn you it may backfire. Conversely, it may also be brilliant. I have never tried peer review before but I’ve spent about a year dreaming up this process. I really wanted to do double peer review, meaning I really wanted students to peer review two of their classmate’s papers but as I tried to schedule it, I recognized in our compressed term, there just isn’t enough time to do that and still allow for the time to do research, the synthesis exercise, and the peer review process. I’ll write a follow up blog after I have trialed the process.
  2. Remember that all writing is contextual. I’ve created this process specific to the needs of my assignment, my course content, and my teaching style. If you want to model what you do after my process, you won’t be able to simply lift my points and my language choices and have a neat fit. Make it your own.

Here is the document describing the process:

Qualitative Paper Assignment

In the document you will find:

  1. My detailed assignment guidelines for the qualitative synthesis paper.
  2. A description of the peer review process.
  3. My content rubric for the paper (out of 33)
  4. The peer reviewed checklist for the students to help them assess their classmate’s work.

As this is an assignment structured using scaffolding theory, these are the basic activities to scaffold the process:

  1. As the course I teach is a research course, I start the term introducing them to the principles of qualitative research. As I teach them about qualitative methods, I am simultaneously connecting the theory content to what they need to write about. They also get a class on searching the literature and what needs to be included in the background to a research study.
  2. I have the students find 5 primary qualitative studies on a topic of their choice. They have to show me these 5 primary studies. There are numerous benefits to this requirement, most of them, selfishly benefit me: a) I ensure that they write this paper using the literature I asked for and not just any old literature. It saves me so much grading time to check this in advance; b) I get to look each one of them in the eyes at least once during this process. For some students, the article checks are the only conversation I’ll have with them all term; c) the students can write their paper knowing they won’t lose marks for incorrectly identifying qualitative studies.
  3. I provide them with an article identification guide to help them distinguish qualitative, quantitative, and discussion papers. Article Type Identification Guide
  4. I have them read their articles next and then we come together and do a synthesis exercise over one class.  I wrote a tweet thread about that exercise a while back (see below).
  5. The students will go off and write and before the planned peer reviewed process they would hand it in to me and I would grade it. The next steps are steps that are untrialed.
  6. Now students will hand in a preliminary paper and share that preliminary paper with a pre-selected classmate.
  7. Each will use the peer review checklist to independently comment on the other’s paper. They will also be welcome to correct grammar and APA.
  8. I will hold a peer review class where they can ask me questions but the goal is also for them to sit down with the rubric for the paper and jointly come up with a score for the content portion of each of their papers. Two chances to practice with the rubric.
  9. Notice that the rubric and the peer review checklist are colour coded. That’s because the colours that match on both documents are the items on the peer review checklist that they will be considering when scoring that section on the rubric. The rubric is broad areas of writing (content, synthesis, research, mechanics) the checklist is ordered by the required heading sections of the paper. Hopefully this will help them realize what expectations affect what sections of the rubric.
  10. After the peer review process, the students will go off and revise their papers based on their discussions. They will hand in a final draft of their paper to me along with a self-scored rubric for their own content.  Their grade will be the score they give themselves — unless they score themselves too low then they will get my score. If they score themselves too high? Well that’s a bridge I’ll cross when and if it happens.
  11. I am not having the students grade each other or assess themselves on APA. They’ll obsess over it and it will take away from what I want them to focus on, which is their content. I’m going to grade their APA.
  12. 50% of their entire course grade goes to the little pieces that make up this assignment. The breakdown of the assignment in terms of value is:
    1. Content of paper 33/
    2. APA 3/
    3. Participating in synthesis class and making progress 3/
    4. Article checks 1/
    5. Peer review 10/

 

How I will score their peer review exactly is still a work in progress and I’m still reflecting on it as I don’t have to “grade it” for a couple months yet. It will be a combination of participating in the peer review, collegiality with peer and being a good citizen and getting documents in on time, a thorough effort at feedback to their peer, the degree to which they pay attention to the feedback they were given in their next draft, a one paragraph response to that feedback, and realistic self-assessments on the rubric.

I’m blessed this term with an abnormally small class which will be extremely useful for me to trial this process and discover the weak spots in my pedagogy.  I hope, those of you who were interested also find this useful. Please feel free to comment below with questions or tweet at me to discuss, or share your own peer review processes.

Publication Peer Mentorship: Starting Your Own Support Group for Writing

 

I have posted on Twitter several times about the publication peer mentorship group I started in my department. Those posts have received a good response and several  inquiries of curiosity which makes writing a blog post about why I started the group and what we do in the group important.

The Background to the Group

First, I should tell you a little bit about our faculty. I work at a non-tenured college in Canada. The college used to be a “community college” once upon a time (people often still call it that, even though the community part got removed from its name before I started working here and that was 15 years ago). About 8 years ago, the provincial government granted the college degree granting status primarily so our department (Nursing) could begin offering Baccalaureate degrees. Prior to that we had a joint program with the University and a 23 month diploma option (which, nursing peeps, if you are cringing at the mention of diploma education, is a really long and sad story). There is one other degree program at the college — Construction Management — and many more diploma and certificate programs. One of the tag lines of my college’s name is “Applied Eduction.” This means we value and focus on practical learning.  We do not have any graduate programs at the college and it is unlikely that we ever will. Being a college that operates in a polytechnique model, my instructor job description does not require a program of research. It doesn’t even require I have a Master’s degree let alone a PhD. Although in my department, a Masters degree is required to be a course leader.

However, being a baccalaureate degree program, our department chair desires that one day we will obtain accreditation from the Canadian Association of Schools of Nursing (CASN). Part of the Accreditation process requires that faculty has a track record of scholarship. Scholarship does not need to be in the form of traditional research but can be any peer reviewed process, so publications, presentations at conferences, and the like.

Of a faculty of about 140, our classroom course faculty is primarily Masters prepared. Course leaders must be Masters prepared but other course instructors do not. There are, however, large number of faculty currently in Masters education. Our clinical instructional faculty are primarily Baccalaureate prepared. We have four faculty members with PhDs (and two of them are our psychology instructors). We have two individuals who have doctoral degrees near completion (one is a doctorate in nursing practice). And then there is me, who is just getting started on the PhD process. I could have floated along until retirement doing what I’m doing and nobody would have cared about the initials behind my name but, for me, a PhD was the next step, and even at 45, I am too young to not do a PhD.

We are a teaching institution and my colleagues focus on and prioritize their teaching above all else. Many of them have no interest in writing, but we are on a push to increase faculty scholarship activities. With a large number of our faculty currently in Masters programs, many of them have papers completed for coursework and papers to write based off of completed Masters theses.  Many of my colleagues also have tremendous expertise in an area of nursing education, clinical education, or nursing practice and they have something to contribute.  My conversations with my colleagues tell me, loud and clear, they feel unsupported and lost as to how to write for publication.

Part of my role in our faculty is to act as writing mentor. I coordinate the assignments across the curriculum in our program. I also offer faculty support with issues related to student writing: plagiarism, grading, rubric creation, and all things APA. I also support faculty with their own writing and, beyond a few Masters assignments, there hasn’t been a big demand for my assistance in this kind of role (which is my way of saying, I should be busier). And even as a Masters prepared faculty member, when it comes to writing and publication and even research, I am one of the most experienced we have.

The Publication Peer Mentorship Group

I was sitting in our Research Coordinator’s office talking about how to motivate scholarship and build faculty capacity to create scholarship, when the idea popped into my head to form a writing support group. I wrote and sent an email to faculty about it within an hour.  Now we are three months in to the group.

Here is what I did:

  1. I sent out an email to all our faculty announcing my idea and seeing if there was any interest. I got probably 4 replies and over the next several weeks, I had 3 more faculty approach me in person.
  2. I started an email list with the interested faculty.
  3. I set a date just before Christmas at a time when most would still be around but wouldn’t be very busy with academic stuff.
  4. We met and we talked.

I’ll be honest. I had no idea what we would even talk about. Someone who had wanted to attend but was unable to asked me to give her a copy of the presentation after. Presentation? That certainly wasn’t what I had in mind. I had started the group but I wasn’t intending to lecture. And I’ll tell you that it is intimidating having everyone look at you for guidance when you feel you are merely baby steps ahead of where they are. The only difference between me and them was that it has never occurred to me to NOT publish what I wrote. Publication to me seemed like the obvious extension to what we do.

There were four of us at the first meeting and everyone had something to actively work on. One person had revisions of her Masters research project to return to a journal. Another had a manuscript she had written for a class ready to submit. Another faculty member wanted to revisit the work she completed as a Masters student. And then myself who was in the position of just having two papers accepted for publication and was awaiting reviewer responses to another two. And I published everything out of my Masters — but that was over 10 years ago and the publication world has changed quite a bit since then. A few others have joined us since then mostly working on new things that will all require literature searches or discussions about approach. Others are in the process of writing up their research thesis and just want to talk about the writing.

 

So how do I conduct the meetings? I wing it. And to be honest, I’m anxious and nervous in anticipation of that. My goal is that the group can be self-sustaining and that I will turn out to be one of many leaders. The most difficult aspect of reaching our faculty members is  that in a culture of teaching, we’ve become accustomed to believing that our lectures, students, our grading responsibilities are our only priority and always come first.  Academic writing guru Jo Van Every (@jovanevery) posted on Twitter a phrase I use regularly. I loosely paraphrase here:

Your students are not more important than your writing, and your writing is not more important than your students. They are equally important and they both deserve to be prioritized.

For example, I have 13 papers left to grade to be done but I’m writing this blog instead. I also have a paper to revise and get back to a publisher and that is going to get some of my time this weekend and next week too. Hence why rule number one is to block off writing time and protect it. It may not be a day, but a couple hours here and there. Set it like an appointment in your calendar and don’t let anything interfere with the time. Writing is hard. Thinking about writing is daunting so it is very EASY to say, “I need to be doing something else,” like grading papers. Chances are that “something else” is in your comfort zone. Even good writers can get pushed out of their comfort zone while facing a blank page.

(Incidentally, I need to write more tweets involving J.K. Rowling because that one was pretty popular)

I try to keep the meetings under an hour (half hour preferably) and everyone must walk out of the room with a plan to get something done, no matter how small that something is. Here are some of the topics we have addressed at the meeting. All these topics have been spurred on by the needs of the members present. I start each meeting with everyone reporting on progress of their goal or identifying a goal they have. These goals and progress reports have inspired many discussions. The goals are also intended to help with accountability.

Topics for discussion:

  1. How to pick a journal to submit to.

  2. How to react or respond to reviewer comments.

  3. I’ve talked about my own publication experiences, of which certainly I have not experienced every possible scenario out there but in my 12 publications and submissions, I’ve had a lot of interesting things happen.

  4. Writing topic ideas suitable for publication and where they come from.

  5. Literature searching and libraries (as struggle for us — our library is funded to support diploma education not baccalaureate).

  6. Creating a presentation first and using that as an outline for a paper.

  7. Finding the right voice for your work.

  8. Sometimes we just talk about how hard it is to write.

  9. Sometimes we talk about how hard it is to find time.

Topics specific to the act of writing are the ones I like talking about the most. One of my colleagues is writing her qualitative findings for an amazing project about the cancer care experiences of rural patients. She has been emotionally invested with her subjects and this project is such a passion for her. Her feedback from her advisor was that her writing was stilted and read like a clinical analysis found in a chart. She needed to come and talk and just get permission to put herself in the work. The writing she needed to do felt strange to her. We’ve had objectivity and anonymity beaten into us through most of our academic schooling.

The main thing I have learned (or rather confirmed) is that faculty writing self-efficacy, especially in faculty with teaching focuses, is a significant issue. If imposter syndrome is bad in tenured faculty, believe me, it engulfs faculty who have been cultured to believe they are lesser and that their ideas are less important than those with fancy degrees. It this might be a gender thing as well. Remember, my discipline is nursing. All the members of my mentorship group are women.

I don’t believe you need a lot of writing experience to lead such a group. I have a lifetime of writing experience both creative and academic. I have experience publishing but my experience is by no means extensive. All you need is one person willing to organize the meetings. What happens at those meetings takes care of itself. The goal is to motivate, inspire, and keep interested writers accountable — not unlike the fiction and poetry writer’s groups I have belonged to in the past. The point is to get people together to talk about writing and realize that their struggles are not unique.

I would be interested in hearing your experiences? What would you find helpful if you were a member of such a group? If you have such a group at your institution, what are its objectives and topics of discussion? What have been your successes and pitfalls? We are just getting started in our group and have had 3 meetings in total. I am in search of inspiring ideas. I’ll be honest, my greatest fear is that I will set a meeting and people will stop comings. So far, though, so good.

Addendum:  I found out just now, I broke the first AND second rules of writing club.

 

The Evolution of My Writing

I used to write my papers on an old Hewlett-Packard word processor.  It had a little tiny memory disk that was about half the size of a hard “floppy” disk, although I don’t think those existed at the time. It was a Christmas gift or a birthday gift from my parents during my last year of high school. I used it through my first 5 years of university, so about 1989-1994.  It had a little tiny computer screen which allowed me to see about 6 lines at a time.  It could be used like an electric typewriter as well. But I liked that I was able to save my writing to these little disks and when I wanted to print something, it auto typed it out like a regular typewriter would at about 80 to 100 words per minute.

Back then I always hand-wrote the first drafts of my papers.  I liked to be able to see a page worth of work at a time and manually flip back and forth between those pages. One hand written page, roughly, worked out to be just short of one double spaced, 12pt font, typed page of writing.  After I finished the first handwritten draft (where I edited little), I used to type the paper into the word processor and then do all my editing there.  The act of typing out what I had written allowed me time to reflect upon some of that first draft uglies and where I needed to make global changes.  Thus, I’ve always believed that the real writing happens in editing.

I wrote my papers using this process through two university degrees and continued to handwrite rough drafts even when I retired the HP and began using Microsoft Word on a PC. Then sometime during the first year of my master’s degree I started writing papers directly into the computer. Papers got harder to write, at that level, and I found I couldn’t write anything in any planned order any more. Also, with 40 references to incorporate, I could no longer write an entire paper in a single day. It was easier to put a heading into a computer file and leave a blank than it was to guess how much of a blank to leave on looseleaf to accommodate what needed to be said.

I presented at a conference this year and while it really had nothing to do with my presentation, someone asked me if writing processes had changed in the era of computers.  I had been talking about a well-known writing process theory that was developed in the 70s that still seems very relevant today, even though fewer people likely handwrite their drafts than would have been common back then.

Now that I write solely on a computer, I still write a pretty rough first draft  but I edit as I write more than I used to.  Autocorrect helps with my clunky spelling skills. But it is still always my goal to get a sketch of what I want to say down in writing and then work my magic while editing. I consider my first drafts so rough, that I would never show a first draft to anyone…. not even co-authors. I always like to let a manuscript germinate for a while. A week is awesome, if I have it, but 2 days is often enough before I tackle the next draft.  I spend the time between reflecting on what I wrote and things I forgot to say.  I look for ways to solidify my argument or make connections between points I may not have seen the first time around.

I’ve heard of these strange creatures who obsess over every word in a sentence and write a clean rough draft that needs nothing more than a polish.  I am not one of them.  And to be honest, I don’t envy them. It would be painful writing that would make me feel like the end was so far away. I believe the key to being prolific is to spew it all out in one shot as quickly and as horribly (or not so horribly) as possible. If you aren’t solid on your thinking in some areas, what you write later may be a clue to how to fix it. I’ve written articulate phrases like ….. and blah blah blah prediction …. into my papers just to remind me what I need to learn more about when I come back and edit that section. Sometimes you have to stop and read more to figure out what comes next.

And that is one thing that has never changed about my writing process. I love revisions. I have always worked better building off an existing idea than staring at a blank page hoping  that words will come. A done first draft feels like a finished paper to me because I know how it begins, I know how it ends, and now I can fill in the holes with what I believe to be the best part of writing: rewriting.

Having said that, you need to write the way you write.  It isn’t wrong to be a multi draft writer. It also isn’t wrong to write each sentence as if you’ll never have to change it.  Just write.

*Image: some really bad fiction writing I did when I was about 20. I remember going back and doing the highlighting. It was the only parts I liked.  (My attempts at fiction, back then, never made it to the word processor phase of writing).