On Using the Media to Teach Research Writing and Critical Analysis

When I first started teaching research methods, I would have told you that having an academic writing assignment in my course was a bad fit. The only writing assignments that I had ever heard of being done were an article critique or a research proposal.  A research proposal was above an undergraduate level and an article critique got them to think about the strengths and weaknesses of research but only apply that knowledge to one isolated study.

However when our program was redesigned, my research methods course was updated to  include teaching the concept of scholarship in nursing. While I wasn’t looking to steer undergraduates toward publication by asking them to write, I did want to get them deeper into the scholarship of research. I had been having them analyze portions of research articles since I began teaching the course, but wanted to push it a step further and have them begin to critically analyze a small body of research on the same topic.  Imagine that! I wanted them to actually read research in a research course.

The First Assignment Plan

My first academic writing assignment plan asked them to find 5 primary studies on a researchable topic of their choice. I told them it was preferable if they chose 5 qualitative or 5 quantitative studies and not try and mix and match the paradigms, although I would make exceptions depending on topic. Hint: it really only works if their quantitative studies are purely descriptive. Undergraduates trying to compare the results of 4 randomized trials and a qualitative study is disastrous… one of these thing is not like the other.  They had to provide a background, nursing significance and explain the gap in knowledge on their topic as indicated by their 5 studies (and other sources as required). A compare and contrast of the findings of the 5 studies and a discussion of study limitations as indicated by the authors of those studies, was the bulk of the 5-7 page limit requirement. Their conclusion to their paper was the purpose statement for a future “hypothetical” study.

What followed, and ultimately what got me to modify the assignment, were 4 appendixes describing portions of a possible future study. Replicating a study in their pile was allowed but “difficulty points” (ultimately meaning I was more forgiving if they made mistakes) were awarded to students who developed the next steps in the body of the work, corrected the flaws in the studies in their pile, or developed a new intervention. I told them they had unlimited funds which resulted in more than a few brand new exercise facilities built.  The appendixes were to describe their study method (A), their sample (B), procedures for ethical protection of their participants (C), and to find and discuss the characteristics of one questionnaire  to measure a one variable, if their study was quantitative, or develop 4 open ended research questions if qualitative (D).

What happened was amazing. The students developed a great understanding of the methodologies they were writing about and using for their hypothetical study design. But the flaws were also huge. The ethics section became repetitive from student to student and the only students who did well on the questionnaire section were the ones who came to see me about their paper and we sat together to identify its characteristics and find reliability and validity information.

And there was one other, not insignificant problem — the assignment was labour intensive to grade. Each paper was 10-15 pages long including the appendixes.

Modifying the Plan: Research Versus the Internet

The decision to modify the paper was as a result of reading this Deitering and Gronemyer (2011) article describing the importance of getting students involved in the public discussion about research. Their arguments about why students should be examining more than scholarly work were compelling and I made them discipline specific by recognizing the following: Students have a need to understand the literature that influences public debate and the opinions of their patients. They are also in need of understanding the difference between a published study and a blog post/newspaper article related to the published study and be able to explain why the former is a more reliable source of evidence.

Let me state the obvious — obvious to us academic-types anyhow — the media gets health information wrong much of the time.  Media articles are rarely written by knowledgeable health care professionals or researchers and are instead written by journalists with no medical training who may never read the full research study but produces an article for their publication based off a press release which may give an incomplete picture. For example, this popular media story on wine drinking and its equivalence to an hour worth of exercise was conducted on rats who drank the human equivalence of 100 bottles of wine to show the effects the media article claimed.  The media report was so much fun it was emailed among my colleagues and we giggled about it wishing it were true (and later that term, a student tackled this topic for my scholarly paper assignment). Sigh… back to the treadmill.

I didn’t want to lose what I had observed to be the best part of the assignment — sending students off to search for 5 primary studies on the same research topic. While settling on 5 as the appropriate number of studies to locate was somewhat of a guess, it proved to be exactly the right number to be challenging for undergraduates, yet possible for most every topic. Some students were able to find more than 5 and have the flexibility to exclude an article that didn’t fit well with the others. Other students struggled to find that 5th article and developed some clever research skills as a result.  The requirement that they show me their 5 primary studies to ensure they were using the correct sources got me face to face with every student. I heard their stories of why the topic interested them. We had conversations on making the articles fit together for easier critical analysis. To this day it surprises me when grading the papers, how much I’ve learned about each student simply from these 5 minute conversations.

While the option to choose their topic based on the inspiration from a media, blog or other internet source remains the most popular option, two other routes to inspiration have also been observed as successful adaptations. Students can also describe a clinical experience they had where the practice they observed differed in some way from the practice they were taught — for example, alcohol swab versus no alcohol swab prior to glucose monitoring.  Personal experiences with the health care system have also been addressed such as a student who gave birth to a premature baby who felt her husband was ignored during kangaroo care. The point is to describe a practice, or relate what the media is saying to influence their readership, and see if the research literature matches the message. It works well with qualitative article sets as well, for example the media article of inspiration can be any first person narrative of their experience with a disease or other condition related to health.

The Assignment Guidelines

In brief, each student paper must contain a discussion of the following:

  1. Background to their research problem.
  2. Significance to nursing. Students are more than welcome to use research from other disciplines to support their research question but they have to explain (and cite) why nurses should care about that problem.
  3. A summary of their media article, clinical scenario or personal experience.
  4. Compare and contrast their 5 primary studies in terms of their research findings.
  5. Discuss the strengths and weaknesses of the research studies reviewed.
  6. Discuss how the research findings compare with the media description (I’ve had students integrate this requirement throughout their compare and contrast and that works also).
  7. Practice implications. In what way, if any, should practice change given what the research says?
  8. A table summarizing their research studies with author, title, independent variable, dependent variable, population, sample size  for quantitive studies, and phenomenon of interest replacing the IV and DV in qualitative sets.

Students are able to construct their paper in the order of their choosing and in whatever way feels logical for their topic.

Evaluation

This assignment has been successful beyond my wildest dreams. Student engagement has increased. I get the chance to speak to every student about their topic which meets my requirement of looking them all in the eyes during the writing process. I strongly believe that having every student connect with you during the writing of their paper reduces the likelihood that they will plagiarize. By having a quick review of their primary studies, I also save myself a tremendous amount of time while grading because I know I am reading a paper amalgamated from the correct type of literature.

In the second year of implementation, due to feeling I was doing a little too much handholding in helping them identify qualitative versus quantitative research, I created a document which outlined the characteristics of various published articles (primary quantitative, primary qualitative, research protocols, review articles, discussion articles, and other). The students are now required to review this document and fill in a log sheet which attempts the article identification prior to approaching me to approve their articles. I no longer identify what their articles are for them which allows me to spend more time exploring how well their articles fit together as a package.

Implementing this paper is time consuming. The mini meetings I have with them about their topic take up all my mid class break time and I usually require another 20-30 minutes immediately preceding or following class to attend to every student who wants my attention on a given day.  I have to teach them how to synthesize their results and thematically analyze the themes from qualitative studies. But the upfront time I put in prior to the paper due date, is time I don’t have to spend frustrated with poorly conceived assignments while grading.

If you are interested in modifying the assignment for use in your own courses, you have my permission. I believe this assignment can work in multidisciplinary contexts. Just send me a tweet to let me know @academicswrite and I’ll be happy to answer questions.

10 Reasons Why Your Students’ Writing is Bad — Part I (1-5)

I said here that I didn’t feel my student’s writing was really all that bad at all.  OK, I’ll admit, sometimes it’s bad. Really bad. Fortunately, the truly bad ones are few and far between. As multifactorial as writing is, so many problems can contribute to the perception that a paper is “bad” and not all of them are the fault of the student. I’ve read papers with poor grammar that are well researched and cohesive. I’ve read papers that are technically well written, but are poorly organized, not on focus, or skim the surface. Regardless there are always steps we can take to make sure the problems of bad student writing, at least those within our control (as some of them will be), are addressed before we have to grade their submissions. Whenever I am grading a student’s paper or reading portions of a draft with the student in my office, I have to come up with a diagnosis for why the paper isn’t working. Many of those diagnoses are preventable problems. Much of what I am about to write I draw from my own experience.  Some of what I write, if you prefer a theoretical perspective, was inspired from the writing theory of John R. Hayes.  Hayes doesn’t make these exact points in his theory but he certainly alludes to many of them.

In Part I of this blog I will examine the causes of bad student assignments that are under the control of the teacher via assignment creation, attitude, or instructional method.  In Part II, I will cover the causes that require student intervention.

(n.b. the image that tops this blog is a doodle one of my students did during an exam she was writing for me. If I could name her, I would. But I can’t. Just know she is talented and self-aware, if not, grammatically perfect.)

  1. Other Priorities

Your academic assignment is in heavy competition with the assignments in the four other courses that the students may be taking, Netflix, social media, their friends, their family demands, Pokemon Go, their jobs, and any infinite number of competing distractions.  Students simply may not do their best work because they are in survival mode and the work you assigned didn’t make the top of the list.

No matter what we do, or how much we may rant, the student has the right to choose what they put their most and least effort into.  If it happens to not be your assignment, don’t take it personally.  You will simply grade it accordingly. However, a relevant assignment that ignites a student’s passion may give the work you assign an edge.

2. Lack of Relevance

Of course you make awesome assignments and your students will be passionate about completing them. A great teacher likely creates the assignment that, if they had to sit through their own course, they would love to write themselves. A great writing assignment will take into consideration the key learning objectives of an entire course.  But I’ve talked to many students about assignments they bring me from courses I am not teaching and sometimes I find they don’t see the relevance. Sometimes, sadly, I read the assignment guidelines and I don’t see the relevance either. We get so immersed in our own courses we think the relevance of our assignments to the big picture is obvious.  But our students are immersed in many courses simultaneously all containing unfamiliar new knowledge, so relevance is more elusive.

Don’t take for granted that your students can see the relevance of your writing assignment. Tell them the relevance. Tell them often — like a broken record often. And if you can connect the relevance of your assignment within every individual class that you teach, even better. A writing assignment needs to be more than a description in a syllabus or in handouts that are rarely mentioned in class.  You must teach your assignment. The fit of the assignment to the big pictures needs to be obvious and the reason why they need to write to learn, rather than do a presentation or an exam, also needs to be obvious.

3. Topic Boredom 

The worst academic writing assignments I ever wrote as an undergraduate student were the ones where every student was writing a variation of the same thing. They were boring assignments where we had to write what the teacher wanted to hear or risk a bad grade.  Creativity in these assignments was always perceived negatively.  If a student can’t find a way to connect with the material they are writing about using their own voice, style, opinion or creative touches, they will likely write a substandard paper. For good writing to take place, the writer has to care about what s/he is saying.  I once investigated a suspected plagiarism case where the student had written a paper for me the term before and earned an A.  The paper I was investigating was so badly written I had to re-look at my file of her paper from the previous term to be sure I hadn’t been mistaken.  Nope. It was still an A. It was not a perfect effort but it was very well done.  What happened?  The paper the student wrote for me was on a topic she personally connected with. The paper she wrote for my colleague was something she didn’t care about or see the relevance of writing.

(The paper wasn’t academic misconduct, by the way, it was just lazily written using frequent short cuts and bad paraphrasing. It was however, an undoubtably bad paper.)

Although the needs of every academic assignment will differ, giving students complete control over their topic choice is the best option, when possible. If there needs to be a finite choice of topics provided then give the students several to choose from. If possible, allow flexibility in how they manage the information for that topic.  Worse than having no choice at all, is the academic writing assignment with guidelines that are nothing more than a fill in the blank questionnaire with heavily pre-scripted points to be made on a topic.  The students will be bored writing it, and you’ll be more bored reading it.

4. Grading Negativity

I often feel I am the only teacher with a writing assignment in my nursing course where I feel impressed with how well my students tackle the task. I often hear from my colleagues that student writing is globally bad.  They were all terrible. One teacher said to me this past year. And when I asked about the grades, they spanned the spectrum of A+ to F.  They weren’t all terrible. They were, as they should be, “normal.”  The bad papers can stick with you like sap on a windshield while you forget those glimmers of hope and moments of brilliance. (Or if they were really all bad, it was likely due to another problem on this list).

Remember the normal curve? Embrace it.  As much as we would like all students to write papers that were A+ quality, (wouldn’t that be nice!), it just isn’t going to happen. Don’t fixate on the bad ones.  Unless you are failing to explain your assignment in some major way, bad papers are likely not your fault.

5. Problems Understanding the Instructions

Assignment guideline clarity is critical to receiving good papers. Some say too much information is baffling but I find that more is better than less.  In fact, I usually find ways to explain the assignment guidelines multiple times.  But more critical than that, students often show evidence of not understanding what would seem to us like obvious terminology.   Describe vs. Discuss.  Critique vs. Analyze. Reflect vs. Detail. It is not uncommon to receive a paper where you asked the student to critique a source, and receive 30 assignments that use description only.  In one of my assignments, students seem to have trouble with understanding the difference between nursing significance (why should we care about this problem?) and nursing implications (how should practice change based on our knowledge of the evidence?). A good proportion of the class thinks they are the same thing. I’m working hard to reduce that misperception.

Ensure that your students are clear on what they are being asked to do. One of the best ways to guarantee clarity is to provide sample assignments written by other students. If you work with large class sizes (like I do), not every student is going to approach you for help and if they all did, you’d be working 24 hours a day. Listen for misinterpretations of an assignment guideline during the conversations you do have. There may be a better way you can explain it.  Watch when grading for common flaws in execution of a particular content item or focus. In the example I give above, many students would combine the nursing significance and nursing implications content into one heading, often inappropriately. I now make sure I take every opportunity to explain the difference. If a good proportion of the class makes the same error, make a clarification the next time the guidelines are presented to a group.  There are teachable moments everywhere.

Proceed to Part II …..