On Using the Media to Teach Research Writing and Critical Analysis

When I first started teaching research methods, I would have told you that having an academic writing assignment in my course was a bad fit. The only writing assignments that I had ever heard of being done were an article critique or a research proposal.  A research proposal was above an undergraduate level and an article critique got them to think about the strengths and weaknesses of research but only apply that knowledge to one isolated study.

However when our program was redesigned, my research methods course was updated to  include teaching the concept of scholarship in nursing. While I wasn’t looking to steer undergraduates toward publication by asking them to write, I did want to get them deeper into the scholarship of research. I had been having them analyze portions of research articles since I began teaching the course, but wanted to push it a step further and have them begin to critically analyze a small body of research on the same topic.  Imagine that! I wanted them to actually read research in a research course.

The First Assignment Plan

My first academic writing assignment plan asked them to find 5 primary studies on a researchable topic of their choice. I told them it was preferable if they chose 5 qualitative or 5 quantitative studies and not try and mix and match the paradigms, although I would make exceptions depending on topic. Hint: it really only works if their quantitative studies are purely descriptive. Undergraduates trying to compare the results of 4 randomized trials and a qualitative study is disastrous… one of these thing is not like the other.  They had to provide a background, nursing significance and explain the gap in knowledge on their topic as indicated by their 5 studies (and other sources as required). A compare and contrast of the findings of the 5 studies and a discussion of study limitations as indicated by the authors of those studies, was the bulk of the 5-7 page limit requirement. Their conclusion to their paper was the purpose statement for a future “hypothetical” study.

What followed, and ultimately what got me to modify the assignment, were 4 appendixes describing portions of a possible future study. Replicating a study in their pile was allowed but “difficulty points” (ultimately meaning I was more forgiving if they made mistakes) were awarded to students who developed the next steps in the body of the work, corrected the flaws in the studies in their pile, or developed a new intervention. I told them they had unlimited funds which resulted in more than a few brand new exercise facilities built.  The appendixes were to describe their study method (A), their sample (B), procedures for ethical protection of their participants (C), and to find and discuss the characteristics of one questionnaire  to measure a one variable, if their study was quantitative, or develop 4 open ended research questions if qualitative (D).

What happened was amazing. The students developed a great understanding of the methodologies they were writing about and using for their hypothetical study design. But the flaws were also huge. The ethics section became repetitive from student to student and the only students who did well on the questionnaire section were the ones who came to see me about their paper and we sat together to identify its characteristics and find reliability and validity information.

And there was one other, not insignificant problem — the assignment was labour intensive to grade. Each paper was 10-15 pages long including the appendixes.

Modifying the Plan: Research Versus the Internet

The decision to modify the paper was as a result of reading this Deitering and Gronemyer (2011) article describing the importance of getting students involved in the public discussion about research. Their arguments about why students should be examining more than scholarly work were compelling and I made them discipline specific by recognizing the following: Students have a need to understand the literature that influences public debate and the opinions of their patients. They are also in need of understanding the difference between a published study and a blog post/newspaper article related to the published study and be able to explain why the former is a more reliable source of evidence.

Let me state the obvious — obvious to us academic-types anyhow — the media gets health information wrong much of the time.  Media articles are rarely written by knowledgeable health care professionals or researchers and are instead written by journalists with no medical training who may never read the full research study but produces an article for their publication based off a press release which may give an incomplete picture. For example, this popular media story on wine drinking and its equivalence to an hour worth of exercise was conducted on rats who drank the human equivalence of 100 bottles of wine to show the effects the media article claimed.  The media report was so much fun it was emailed among my colleagues and we giggled about it wishing it were true (and later that term, a student tackled this topic for my scholarly paper assignment). Sigh… back to the treadmill.

I didn’t want to lose what I had observed to be the best part of the assignment — sending students off to search for 5 primary studies on the same research topic. While settling on 5 as the appropriate number of studies to locate was somewhat of a guess, it proved to be exactly the right number to be challenging for undergraduates, yet possible for most every topic. Some students were able to find more than 5 and have the flexibility to exclude an article that didn’t fit well with the others. Other students struggled to find that 5th article and developed some clever research skills as a result.  The requirement that they show me their 5 primary studies to ensure they were using the correct sources got me face to face with every student. I heard their stories of why the topic interested them. We had conversations on making the articles fit together for easier critical analysis. To this day it surprises me when grading the papers, how much I’ve learned about each student simply from these 5 minute conversations.

While the option to choose their topic based on the inspiration from a media, blog or other internet source remains the most popular option, two other routes to inspiration have also been observed as successful adaptations. Students can also describe a clinical experience they had where the practice they observed differed in some way from the practice they were taught — for example, alcohol swab versus no alcohol swab prior to glucose monitoring.  Personal experiences with the health care system have also been addressed such as a student who gave birth to a premature baby who felt her husband was ignored during kangaroo care. The point is to describe a practice, or relate what the media is saying to influence their readership, and see if the research literature matches the message. It works well with qualitative article sets as well, for example the media article of inspiration can be any first person narrative of their experience with a disease or other condition related to health.

The Assignment Guidelines

In brief, each student paper must contain a discussion of the following:

  1. Background to their research problem.
  2. Significance to nursing. Students are more than welcome to use research from other disciplines to support their research question but they have to explain (and cite) why nurses should care about that problem.
  3. A summary of their media article, clinical scenario or personal experience.
  4. Compare and contrast their 5 primary studies in terms of their research findings.
  5. Discuss the strengths and weaknesses of the research studies reviewed.
  6. Discuss how the research findings compare with the media description (I’ve had students integrate this requirement throughout their compare and contrast and that works also).
  7. Practice implications. In what way, if any, should practice change given what the research says?
  8. A table summarizing their research studies with author, title, independent variable, dependent variable, population, sample size  for quantitive studies, and phenomenon of interest replacing the IV and DV in qualitative sets.

Students are able to construct their paper in the order of their choosing and in whatever way feels logical for their topic.

Evaluation

This assignment has been successful beyond my wildest dreams. Student engagement has increased. I get the chance to speak to every student about their topic which meets my requirement of looking them all in the eyes during the writing process. I strongly believe that having every student connect with you during the writing of their paper reduces the likelihood that they will plagiarize. By having a quick review of their primary studies, I also save myself a tremendous amount of time while grading because I know I am reading a paper amalgamated from the correct type of literature.

In the second year of implementation, due to feeling I was doing a little too much handholding in helping them identify qualitative versus quantitative research, I created a document which outlined the characteristics of various published articles (primary quantitative, primary qualitative, research protocols, review articles, discussion articles, and other). The students are now required to review this document and fill in a log sheet which attempts the article identification prior to approaching me to approve their articles. I no longer identify what their articles are for them which allows me to spend more time exploring how well their articles fit together as a package.

Implementing this paper is time consuming. The mini meetings I have with them about their topic take up all my mid class break time and I usually require another 20-30 minutes immediately preceding or following class to attend to every student who wants my attention on a given day.  I have to teach them how to synthesize their results and thematically analyze the themes from qualitative studies. But the upfront time I put in prior to the paper due date, is time I don’t have to spend frustrated with poorly conceived assignments while grading.

If you are interested in modifying the assignment for use in your own courses, you have my permission. I believe this assignment can work in multidisciplinary contexts. Just send me a tweet to let me know @academicswrite and I’ll be happy to answer questions.