The Consistency Conundrum Part I: Is Inconsistency in Writing Instruction Really a Problem?

Posted on by academicswrite

Many many years ago when I was a new nursing instructor and I was getting assigned to the things that nursing instructors get assigned to do, I was, for a few years, plunked in the skills lab for our spring session course.

We got the same instructions every orientation from the instructors assigned to organize the course. “Every person in this room, has to teach the skill in exactly the same way.” 

And we’d all nod our heads and go off and teach it the way we knew how to do the skill. I guarantee we all THOUGHT we were teaching it the same way because, it’s a skill. It’s objective. There is only one way to do it, right? 

My way. The right way. Which is the same as your way. Of course. 

Why were we demanding skills consistency? I don’t remember any reason beyond the fact that it was students’ number one complaint on their course evaluations. What they were taught seemed to them to be very different from teacher to teacher. 

We weren’t consistent. 

Nearly 15 years later, we haven’t solved that problem. They still complain of inconsistency in skills instruction.

We never really got specific information about what part of the psychomotor activity wasn’t consistent from teacher to teacher, because psychomotor skills in nursing are really a set of safety and infection control principles that need to be adjusted slightly for context, but it seemed their consistency issues were as nuanced as hand position when holding the forceps that were holding the sterile dressing; or — here is a timely one — do you put your mask on before you put on your sterile gown and gloves last, or some other order? And does it actually matter? How drastically different could our demonstrations be from one teacher to another? 

Why do we worry about consistency at all?

I began to question, why then, if it was not possible for instructors to present a guise of consistency with a psychomotor skill, why did we worry about consistency at all? What prompted complaints of inconsistency? And wasn’t consistency just about different teaching styles? The inconsistency problem forced me to sit through countless overly long skills demonstrations by our course leader attempting to get us to all agree on the procedure.

What resulted was 8 instructors interrupting her, Well, at my hospital, we do it this way. 

I hold a pretty constant stance of being norm-critical, even back in those early days of my teaching career. I can remember, in an attempt to put myself out of the constant misery of the consistency meetings, speaking up and saying something like, “Perhaps, we should instead teach students to embrace the ambiguity. They need to learn to be adaptable and recognize when principles are being applied instead of obsessing over minor differences in technical performance.”  

I think in some circumstances, inconsistency can be a real problem – as I complain about it incessantly every time I submit an expense claim at my institution and the rules are different, yet again. I can understand why in a lab experiment, you have to use the exact same number and drops of chemical using the same dropper in each replication. But in the real world we are dealing with independent thinking human beings. For the most part, what we and students call inconsistency is really just a euphemistic way to say, I feel things are unfair. It is probably a genuine lack of fairness — as recent events have taught us, inequities are perpetual in higher education.

That lack of fairness, however, in my opinion, is born out of insistent adherence to rigidity and black and white thinking in a context that is nuanced with situations that are never identical from person to person. And of course, rules always favour the privileged.

And here is a key point: students don’t complain about inconsistency unless they feel it has caused them personally to lose marks or be evaluated differently in some way. 

Transferring consistency practices to writing instruction

Now let’s talk about what I am really here to talk about – consistency in writing instruction and grading. I mean, yeah, writing is not a psychomotor skill.  This set of blog posts is inspired by a Twitter thread I wrote on the topic on the weekend. Then the US election was called and the thread sort of sunk into oblivion, but I feel I had more to say on the subject. It also ties in nicely with my PhD thesis work, so, hence, I blog.  

You see, I believe the problem with psychomotor skill inconsistency wasn’t really with inability to tolerate minor differences in skill performance between instructors, it was more to do with inability to tolerate perceived differences in how they were evaluated on those skills. It was about grading. It was about feeling criticized when a teacher made a suggestion to change a hand position or do something in a different order which was not EXACTLY how they’d been taught. Those suggestions might have been about efficiency rather than evaluation, but – and this is a huge problem in clinical education – it’s really hard for a student to tell the difference between evaluation and just a suggestion when they always feel like they are being evaluated.  

They have the same complaints about writing. They could tell me about all the things in their writing lives they felt were unfair.

“I had to use her words, her way, her everything,” one student told me during the interview phase of my research. “Otherwise I got a bad grade.”  

Another student told me about getting a bad grade because they didn’t have a clinical experience that particular term that fit neatly into the assignment guideline instructions. This student asked the teacher if they could use an alternative — an experience from a previous term. The teacher refused to allow adjustments. The student, then, could only artificially address some of the required aspects of the writing assignment. No surprise, assignment comes back with a reduced grade.

These examples are of teacher behaviours that overgeneralize perceived rules that don’t take context into consideration. The first student felt like her writing voice was not valued. The second student didn’t understand why an alternative experience that was a better fit for the assignment, couldn’t be used. What did it matter what clinical rotation the experience was drawn from?

And both students could tell me stories of the opposite happening to them or other students with the same or different teachers — flexible teachers — which is what makes observing differences in instructor responses feel all the more unjust when you are the student forced to artificially fit your work into some tight boundary. There was no consistency.

What happens when we attempt to fix inconsistency with standardization?

So, what tends to happen when we attempt to respond to student complains of inconsistency in our programs?  We standardize things. We standardize things because we think its like having to use the same dropper in all replications of a chemical experiment — it is more rigourous. Sometimes we implement standardization practices on our own and sometimes it is handed down to us from administration – “everyone will use this standardized rubric for academic papers.” We give more instructions instead of less. We remove choice. We prescribe the exact details of what we expect students to say in their papers to the point that the guidelines are longer than what students are expected to hand in.  

In my doctoral research, students described a really rigid writing environment. They told me that writing in nursing was actually “pretty basic,” “fill-in-the-blank,” everything was right there in the rubric – put this, this, and this, for 5 points, no wiggle room for creativity. I’m loosely typing their exact words. All 20 students in my study described some kind of rigid context for writing in at least one nursing classroom which suggested to me that a push toward standardization is the norm rather than the exception.

And, guess what — this won’t be a shock — they still complained of inconsistency. You can read more about what my students said about the writing context specific to nursing education in my previous blog: It depends on the professor. 

Faculty in the two nursing programs I sampled from in my study, turned to what they thought to be an objectivist solution – standardization, very specific rubrics, identical fill-in-the blank type writing assignments – and it did not solve the problem of inconsistency.  Consistency is really another word for reliability. The lack of reliability in grading was both inter- and intra-rater. The perception of inconsistency existed from student to student whether those two students were being graded by the same teacher or different teachers. 

Herein lies the paradox: In addition to still complaining about inconsistency, some students also resented being hemmed in. They resented that they couldn’t write about the things they were interested in. It made them put less effort into writing and researching – “it will be the first sources I find…. BOOM…. That’s what I’ll use to write my paper.”  (Again, loosely quoted, except for the BOOM).

So standardization kills exploration, curiosity, creativity, innovation, independent thinking. It also trains students to expect you to deliver assignments that provide them with 100% certainty and eliminates all the ambiguity from the learning environment. Is that a worthy trade off? Especially when that standardization has essentially no effect on the problem it was designed to solve in the first place… inconsistency.  

Like using the same dropper in a chemistry experiment, standardization attempts to set up equal conditions for all students. It professes to increase fairness. But we know students aren’t all standing on equal ground. My data showed that privileged students – students with previous educational experience or who had family members who understood higher education – had no problems adjusting from teacher to teacher and their different rules, pet peeves, and biases. They had the skills and knowledge of academic life, to do that and accepted it as normative. But international students, Indigenous students, and students from first-gen and disadvantaged backgrounds felt they could never figure it out their teacher expectations. To them, the so called rigid rules were a bunch of senseless arbitrariness. 

Standardization, privileges the already privileged.  

Why do we keep thinking standardization works? 

I sense we are uncomfortable with our students’ discomfort so we are driven to eliminate it, even though learning is not supposed to be all comfort and roses. I also think standardization is implemented to serve more than just the reliability problem. It also is implemented to quell the anxieties among faculty who are not comfortable assigning and grading writing.  Maybe they feel assigning the same fill-in-the-blank style “essay” (I use that word very lightly here) on the same topic to every student will help them better compare one student to another and more fairly assign grades? I’ve also heard the argument from some faculty that they have to control the topics students choose because they don’t want students choosing a topic they don’t know much about themselves because they won’t know if the student addressed the topic correctly. 

But all of this is anecdotal. There is not much research exploring what drives faculty decisions in creating writing assignments in nursing…. Or in other disciplines for that matter. 

Needing to control the content is a sign of anxiety as well. It’s like faculty doubt their own skills to be discerning readers who are capable of recognizing clarity in text and engaging writing. Faculty, as consumers of academic material, have the ability to know when they are learning something, when ideas are presented effectively, and when something seems off or improbable in the information given. 

Anxieties like this are not uncommon among faculty in nursing programs. Most nursing instructors are not PhD prepared. Nursing has not yet reached that state of PhD prepared surpluses present in most other disciplines. Most individuals teaching undergraduate courses in nursing are Master’s prepared. They also tend to be more comfortable in the nursing practice area rather than the academic arena. We need to better understand their learning needs for writing instruction. 

Flexibility breeds perceptions of consistency 

I think we’ve been thinking about the inconsistency issue all wrong. Structuring and standardization makes complaints of inconsistency increase through providing a false sense of objectivism. Standardization isn’t really standardization at all when everyone reads the so-called standard and develops a different interpretation for defining what is a correct representation of that standard. 

Standardization then removes validity from the assignment; thus the assignment no longer elicits the learning it was designed to assess. Students are so focused on rules and APA and grades that they no longer think about content or engaging a reader. The reader is the teacher. We instead assess ability to follow rules and deliver what a teacher wants. Students in their course evaluations then question fairness and question the point of writing. The students in my research who could not figure out how to read teachers, resented being hemmed in, and they disengaged with writing.

I’ve often said, we are key contributors to student bad writing by making them write things they don’t care about. How then can flexibility help?

Next: Part II. Looking at using flexible practices to reduce inconsistency.

Also: Watch the video that outlines some of my study findings.

Navigating Undergraduate Academic Writing: Guess What? It Depends on the Professor

Posted on by academicswrite

It happened in one of my early student interviews. The student read the item on my developing questionnaire that asked them to self-assess their ability to adapt their writing to meet the needs of the assignment. The question was supposed to be getting at switching from genre to genre. If they were writing a reflection, did they sound reflective? If they wrote an academic assignment, did they get their point across in the form of an argument and address a specific question? But her response was, “I’m meeting the needs of the professor, not necessarily meeting the needs of the assignment.”

And it kept happening over and over – sometimes with the same question sometimes with other questions – the students in my sample talked about needing to figure out the teacher. It wasn’t just a handful of the students in the sample. It was 100% of the students in my sample. They pondered issues such as: What were her preferences? What biases does she have? Is she an easy or hard grader? How approachable is she? How open is she to re-interpreting assignment guidelines with uncertain students? How well does this teacher actually understand academic writing themselves?

There is a novice to expert continuum with guiding writing as there is a novice to expert continuum with becoming a writer. Students can tell when faculty are not confident in their own skills to assign and evaluate writing – and this state is not uncommon in undergraduate nursing which can be taught by faculty who aren’t required to write and publish themselves. Novice instructors are far more likely to hold tight reigns on their assignments and rubrics and be overly focused on surface textual features.

Navigating the writing context is like a journey down the yellow brick road in search of the great and powerful Oz (the teacher) who can solve all their academic writing needs. When they get to know Oz, they sometimes find Oz is not the all-powerful being that they’ve hoped to find. Oz is flawed and fallible, but Oz does have the power to help students recognize that the true power of being a good writer was within them all along.

Students had various strategies for success in this mission of “figuring out” the teacher such as: paying attention to a teacher’s overt opinions on a subject and matching them; noticing what words teachers like to use and slipping them into their own writing;  talking to their classmates on how they saw the assignment; talking to classmates who had already been through the class to see how they perceived the assignment and the teacher; testing out ideas about topics with instructors and peers; judging their relationship with the teacher to see if she can “take it.”

Other students seemed to never be able to figure out teachers no matter what they did. Or they felt they could learn to read the teacher if they ever got the opportunity to write for them more than once in a term — which rarely happened.  Their biggest faux pas in this area seemed to be that many of these students would never actually approach a teacher to ask questions, or they felt that when they did, their questions were misunderstood or brushed aside. Often these students were the ones that you would most want to come and speak to you – immigrant students, students where they learned English at an older age such as when a teen or an adult, and, in nursing, it was also often male students.  These are our students who perhaps struggle to feel like they belong because they don’t fit the stereotype of who a nurse is – passive, polite, white females. And for those who may be thinking, It isn’t like that anymore — implicitly these traits in nursing are still the most highly rewarded and have an easier journey navigating the nursing world.

The opinionated students don’t fit in well either as they often insist on maintaining their own opinion and making things their own. They are the ones that twist assignment criteria to ensure they can say what they want to say. Sometimes these behaviours are highly rewarded, but other times it means they fail the assignment.

It depends on the professor.

What concerns me most is what this means for writing as a pedagogy. If the strongest students are graded highly because they are able to spit back to the teacher exactly what they want to hear, if the writing that is most highly rewarded is the writing that fits into a nice little box where the number of checkmarks on a rubric is the only indicator of learning and quality, we have a problem. It means higher education is not about independent thinking. It’s not about recognizing where boundaries can be stretched and alternative perspectives are valued.

The undergraduate students in my sample spoke about a very problematic culture of writing in higher education. I will speak broadly to this because while I think nursing has some work to do, I don’t think some of the problems I see in the reports I heard from nursing students specifically are exclusive to nursing contexts. Students spoke about “bullet-point” assignments, “fill-in-the-blank” style assignment guidelines. A few students – and this especially came from students who had experience as students in other disciplines such as psychology, English, or gender studies – recognized that the writing was different in nursing.  It was more “simple.”  It was certainly often inflexible. You must “stay within script”. You can be creative but only “within the dimensions they want us to be.” There are boundaries. Being creative, depending on the professor, could mean losing 10% of your grade. There are far too many assignments where there isn’t enough choice and students are forced to write about topics they do not care about. For those students, in those moments, writing becomes a functional and robotic task.

Why do we not have enough faith in our undergraduates to be able to give them free reign to go where their ideas and passions take them?

Are we afraid that we can’t fairly grade if our 60 students aren’t handing in 60 cookie cutter assignments all answering the exact same question using the exact same literature? Who wants to grade those? We lament grading because of the things we do to ourselves in assignment guidelines. I’m asleep already just thinking about it.

Are we afraid if we don’t know the topic to the same degree an expert would know the topic that we aren’t capable of accurately assessing writing?  I don’t know about that – I know when someone has written something in a way that is accessible to me and when they’ve caught my attention and I learned something. Without knowing much about the topic, I am also capable of being sceptical when the accuracy seems “off” or there seems there is another side of the story that isn’t being presented. I am an astute reader. I assume all faculty are astute readers too.

Do we tighten our rubrics into minute details because then we are able to be more reliable in our grading across students or between raters?  But what are we sacrificing in terms of assignment validity in the process? Are we still able to capture the objectives we are hoping our students leave our classroom with at the end of the term? Seriously, just feed them a multiple-choice test if you want strict criteria and one correct answer.

I’m getting a little ranty. I can feel it.  But this is a hill I am willing to die on.  If you are not giving writing assignments that students can get excited about – and I mean every student – your students are learning you, not the threshold concepts of your subject matter. Students are equating their success at writing with their ability to be successful in navigating the writing context, teacher preference, and their levels of writing self-efficacy are impacted as a result. Papers are not about one correct answer. Writing is not about the text produced, I mean, yeah, you want it to be readable and not “messy” – students get that.  As one of my student participants said, “It almost becomes part of your identity. Like you care more about that subject when you go to other places. And then you’re also looking into the literature, so what do other people know? And you’re sort of building yourself and your knowledge base.”  A student walking away from your course should feel that about every assignment.

This little rant is courtesy of a preliminary analysis emerging from a secondary analysis of Congitive Interviews conducted with the primary objective of editing the developing Situated Academic Writing Self-Efficacy Scale. 

Student Peer Review Process? Here’s My Version

Posted on by academicswrite

When I tweeted yesterday that I was creating a peer review process for my students to assess each other’s writing and provided a sample of what I was up to, I had many requests to share the whole process.

I am happy to do that and will do so here in this post with two caveats:

  1. This process is untested. This means, I can’t even give you anecdotal feedback as to if it works. So for now I will share the materials but it is probably wise to warn you it may backfire. Conversely, it may also be brilliant. I have never tried peer review before but I’ve spent about a year dreaming up this process. I really wanted to do double peer review, meaning I really wanted students to peer review two of their classmate’s papers but as I tried to schedule it, I recognized in our compressed term, there just isn’t enough time to do that and still allow for the time to do research, the synthesis exercise, and the peer review process. I’ll write a follow up blog after I have trialed the process.
  2. Remember that all writing is contextual. I’ve created this process specific to the needs of my assignment, my course content, and my teaching style. If you want to model what you do after my process, you won’t be able to simply lift my points and my language choices and have a neat fit. Make it your own.

Here is the document describing the process:

Qualitative Paper Assignment

In the document you will find:

  1. My detailed assignment guidelines for the qualitative synthesis paper.
  2. A description of the peer review process.
  3. My content rubric for the paper (out of 33)
  4. The peer reviewed checklist for the students to help them assess their classmate’s work.

As this is an assignment structured using scaffolding theory, these are the basic activities to scaffold the process:

  1. As the course I teach is a research course, I start the term introducing them to the principles of qualitative research. As I teach them about qualitative methods, I am simultaneously connecting the theory content to what they need to write about. They also get a class on searching the literature and what needs to be included in the background to a research study.
  2. I have the students find 5 primary qualitative studies on a topic of their choice. They have to show me these 5 primary studies. There are numerous benefits to this requirement, most of them, selfishly benefit me: a) I ensure that they write this paper using the literature I asked for and not just any old literature. It saves me so much grading time to check this in advance; b) I get to look each one of them in the eyes at least once during this process. For some students, the article checks are the only conversation I’ll have with them all term; c) the students can write their paper knowing they won’t lose marks for incorrectly identifying qualitative studies.
  3. I provide them with an article identification guide to help them distinguish qualitative, quantitative, and discussion papers. Article Type Identification Guide
  4. I have them read their articles next and then we come together and do a synthesis exercise over one class.  I wrote a tweet thread about that exercise a while back (see below).
  5. The students will go off and write and before the planned peer reviewed process they would hand it in to me and I would grade it. The next steps are steps that are untrialed.
  6. Now students will hand in a preliminary paper and share that preliminary paper with a pre-selected classmate.
  7. Each will use the peer review checklist to independently comment on the other’s paper. They will also be welcome to correct grammar and APA.
  8. I will hold a peer review class where they can ask me questions but the goal is also for them to sit down with the rubric for the paper and jointly come up with a score for the content portion of each of their papers. Two chances to practice with the rubric.
  9. Notice that the rubric and the peer review checklist are colour coded. That’s because the colours that match on both documents are the items on the peer review checklist that they will be considering when scoring that section on the rubric. The rubric is broad areas of writing (content, synthesis, research, mechanics) the checklist is ordered by the required heading sections of the paper. Hopefully this will help them realize what expectations affect what sections of the rubric.
  10. After the peer review process, the students will go off and revise their papers based on their discussions. They will hand in a final draft of their paper to me along with a self-scored rubric for their own content.  Their grade will be the score they give themselves — unless they score themselves too low then they will get my score. If they score themselves too high? Well that’s a bridge I’ll cross when and if it happens.
  11. I am not having the students grade each other or assess themselves on APA. They’ll obsess over it and it will take away from what I want them to focus on, which is their content. I’m going to grade their APA.
  12. 50% of their entire course grade goes to the little pieces that make up this assignment. The breakdown of the assignment in terms of value is:
    1. Content of paper 33/
    2. APA 3/
    3. Participating in synthesis class and making progress 3/
    4. Article checks 1/
    5. Peer review 10/

 

How I will score their peer review exactly is still a work in progress and I’m still reflecting on it as I don’t have to “grade it” for a couple months yet. It will be a combination of participating in the peer review, collegiality with peer and being a good citizen and getting documents in on time, a thorough effort at feedback to their peer, the degree to which they pay attention to the feedback they were given in their next draft, a one paragraph response to that feedback, and realistic self-assessments on the rubric.

I’m blessed this term with an abnormally small class which will be extremely useful for me to trial this process and discover the weak spots in my pedagogy.  I hope, those of you who were interested also find this useful. Please feel free to comment below with questions or tweet at me to discuss, or share your own peer review processes.

Constructing Writing Practices: A writing model for all disciplines

Posted on by academicswrite

I woke up yesterday morning to an email from an unfamiliar name in my inbox titled, “article you just published.” It was a nurse scholar from Georgetown requesting a copy of a publication I knew was coming soon, but I didn’t know had hit the presses yet. Hot off the press at the Journal of Nursing Education.

Screen Shot 2018-07-04 at 1.03.51 PMconstructing-writing-practices-in-nursing (ooh I hope this PDF works — it may be linked here, and it probably isn’t copyright appropriate but we’ll see how long it lasts)

This is the first time I’ve been emailed directly for an article of mine. But it is also the first time my current work has been published in a journal with a > 2.0 impact factor (high for nursing education journals). And then the ivy league comes calling. Also a first.

I tell the story of the birth of this paper in the article itself and my engagement with the literature to produce it. Believe it or not, this section was requested by reviewers. I think they expected a couple of sentences and I gave them about two pages instead — oh well — be careful what you ask for.

The paper started as a philosophy of nursing science assignment where I was asked to address a controversy in my research area. What immediately came to mind was the deep sense of devaluing of writing in nursing and nursing’s anti academic discourse — both of which contribute to the much talked about theory-practice gap that pervades practice disciplines such as nursing (and most health professions, but also other practice disciplines like education and business).

In combination with the anti academic discourse, I had just spent the fall revising a paper exploring all the writing self-efficacy measurements developed for post-secondary populations through a template analysis of the items on these questionnaires. I was looking to find out the constructs psychometricians were identifying as having influence on writing self-efficacy of students. The largest category of items in the template focused on surface writing elements like punctuation, and putting together a paragraph, or writing sentences with subjects, verbs, and nouns, or can you write clearly, with focus. Those were not the elements of writing that I saw my students agonize over when writing for me. They agonized over topic choices and ideas and understanding what they were reading and how frustrating writing could be. The model that developed from this template analysis was a combination of Bandura’s self-efficacy theory and Flower and Hayes’s cognitive processing model of writing and a reviewer asked me if this was it…. was this template enough to describe writing — and more specifically, writing in nursing? I wanted to address this question.  I had also been simultaneously immersed in the literature talking about writing as a socially constructed process so I also knew the model I would eventually develop would be situated in a socially constructed epistemology.

The components of the model can be defined liked this:

Identity: Incorporates writing voice, the self as it appears within a written text, past experiences with writing and their influence on present writing, and levels of writing self-efficacy. Reflexivity facilitates the metacognition and intertwining required to activate the other components of the model as they relate to writing and nursing identity.

Creativity:Novelty and originality as defined by a discipline inform creativity. Idea generation, synthesis, and interpretive abilities all require creativity. Creativity fuels passions and develops identity.

Emotions:Writing emotions can be positive or negative, are subject to roller-coaster extremes, and will drive or inhibit the writing act. Emotions are present at all phases of writing from planning to feedback.

Relational Aspects: Writers form relationships with the sources they incorporate through citation, inspiration, or interpretation. Writers write for an imagined audience and that audience connects with their writing when a writer reveals themselves in their work. Students also form relationships with their teachers during pedagogical processes and feedback interactions.

Context:The writing context includes perceived difficulty of the writing task and writing evaluators, the stakes involved in producing a well-received product, and the values and demands inherent in a disciplinary discourse.

The paper emerged in four phases:

  1. A two page proposal which focused on the theory practice gap and anti academic discourse. I didn’t know at this phase I would be building a model.
  2. A seminar on my topic where I presented the first drawing of the model based on the layers of a globe. I even had a visual image of that globe which when I shared it with my classmates and asked them to reflect on it and discuss it, really fell flat. They didn’t get it — although I have to say that one of my classmates recently, after writing her candidacy papers said to me, “I totally get this now.” It just takes the right kind of writing experience. Screen Shot 2018-07-04 at 12.49.51 PMI don’t remember combining creative and emotional knowing at this stage. I wonder when that changed? Probably where everything changes: in the act of writing.
  3. The final draft of the paper where I removed the visual drawing from the paper because it hadn’t worked when I presented it to a test audience. The paper just described the model as an intertwined process, with identity at the core, where each one of the any the five factors could be the focus at any point of the writing process or they may be simultaneously influencing one another and merged through reflection.
  4. The post submission review process the article changed again mostly in my discussion of nursing’s relationship to writing. Virtually nothing of the text of the model changed other than the reviewers asked me to attempt to draw the model again. So I did… I drew some rough sketches of the model on my own and then I called in an artist pro (my 17 year old daughter Emma) and asked her to draw me a better version. She was a real pro. She drew me four versions on her digital drawing tablet using my version as inspiration and we ended up combining two together. I liked the angular look she had given one version — the twisted strands of the model that you see with the labels on them. They reminded me of how you wrap a tensor bandage. But I liked the the round twist she put on her rounder version of the model so we combined the two into what you see as the header to this blog. IMG_7725My very rough trial drawing of my vision for my model. I saw the intertwining as a braid. As you can see, Emma’s final version at the top of the blog is just so much more effective.

The model is black and white in the article but for poster presentations I had upcoming I asked for a coloured version. I let her pick the colours. Then with the help of some text templates from @academicbatgirl I decided to make a mug of it.

fullsizeoutput_1b06NSFW — but it will comfort me at home.

I wrote the paper for nursing, prepared the poster for a nursing education conference,  but I decided with a bit of an elbowing from my advisor to enter the poster in the faculty of health sciences poster competition. I had no chance of winning in this biomedical positivist world where most of the work is physiological or microbial or population health so I was curious how the judging sessions went. I ended up with two judges one from microbiology and the other from molecular genetics (hilarious — I don’t even know what this is) and I spend my 10 minutes just talking about academic writing and its genres and I managed to get one of them to say, hey … this isn’t just for nursing, this could work for all disciplines. Getting that statement out of a judges who were very unlikely to share my worldview, was winning enough for me.

This model is what I will use to develop the items to assess writing self-efficacy on a new questionnaire designed from a constructivist perspective of writing. I’ve already developed the items but you know how the PhD process goes — several hurdles to jump over before I can get started on testing the questionnaire.

The paper appears here:

Mitchell, K. M. (2018). Constructing writing practices in nursing. Journal of Nursing Education, 27(7), 399-407. doi:10.3928/01484834-20180618-04

If your library doesn’t subscribe to it and you would like a copy of the article please feel free to email me at academicswrite@gmail.com or contact me on twitter @academicswrite

 

The Value of Valuing Writing Self-Efficacy: Changing thinking

Posted on by academicswrite

If Doctoral programs didn’t change your thinking, they wouldn’t be doing their job would they? Here at the start of a new year, I thought I might reflect upon what has happened to my thinking on my planned thesis project to develop a measurement instrument to assess writing self-efficacy.

I finished all my required course work toward my doctoral degree last month and I’m itching for the next steps.  I still have one more course to take and that is an elective I, and my committee, have agreed upon which will fine-tune my skills in measurement of psychological concepts and the statistics of assessing those measurements.  I’m really looking forward to the change in pace as I have been immersed in philosophical ramblings for quite some time now and that is hard thinking. Something a little more “rule based” and structured might be nice. I say that now but I’ll be frustrated, no doubt, by the particulars in no time. In some ways, taking the course is a bit for show on my transcript so no one questions where I got my measurement training from when I go to defend. I would rather sit and read a hundred articles on my own and figure it out with textbooks and conversations. The bad thing about courses is that the structure I just admitted to craving, hems you in. I really hope I have some flexibility in terms of what I read about and how I tackle my assignments but that is usually not the case.

Since 2011 I’ve been studying writing self-efficacy. I’d like to say I fell into that area of research inspired by something profound I read or a conversation I had but it was quite happenstance and to some degree arbitrary. I had read nothing. I just knew my students lacked self-efficacy about their abilities to write the paper I assigned them. I’ve since read a lot and my thinking has shaped — it is a little less a big lump of clay… it’s taking form. I have opinions. I am developing expertise.

Before I even entered my PhD program I had conducted three studies and a questionnaire review on the topic. I knew when I was writing my please-admit-me letter that I wanted to develop a measurement instrument to measure writing self-efficacy. Nothing about that has changed. I’m going forth. But my thinking about how to approach the project has changed a lot. One of my classmates just asked me recently how it is I’ve managed to get this far and not change my topic.* (She, incidentally, has changed her study focus three times). My response was, first, that it was a bit of pragmatism…. the most direct route to graduation so I could get on with doing exciting and meaningful stuff.

My second response was that it had changed, philosophically. I wouldn’t have considered myself a theoretical thinker when I wrote that admission letter — that turns out to be absolutely not true, and slightly lacking in self-awareness. The originall vision was straight up statistical psychometrics. But, partially because I had to for a course, I developed a constructivist model of writing (for nursing) — bracketed for a reason — which I revised and sent back to a journal at their request over the holidays. But the reading for that has lead to other thoughts about writing self-efficacy, my chosen concept. I chose the concept when I had read nothing but now I have read plenty.

  1. Constructivism is the road to better measurement of writing self-efficacy. Writing has been through three epistemological shifts (product, process, social) that happened in fairly rapid succession and the tools that measure writing self-efficacy reflect that. The earliest tools assessed it by grammar fault and ability to construct sentences and be clear. Later ones took a more cognitive process, motivational, self-regulation, perspective. But none of the tools take a social constructivist perspective. Some of the tools have the occasional item that brushes up against constructivism but they don’t capture all the social aspects of writing bound to affect writing self-efficacy. How do I know this? I did the work and it was published in the Journal of Nursing Measurement along with an accompanying editorial.
  2. Writing self-efficacy does not have as strong of a relationship to grades as we would like to think. I certainly have not seen any convincing evidence in my own studies or anyone else, that it actually predicts grades…. at least not in a real-world relevant way. (In health research they would call what I am talking about clinical significance.) Part of this prediction failure is related to context. People assess their self-efficacy based on previous performance but in the face of a new teacher, a new subject, a new discipline, new rules, they may assess their own ability poorly. I for example would tell you right now that I believe I have fairly strong skills and knowledge of measurement based on the reading I’ve done and my research experience. I should ace my measurement course without difficulty. But I’m walking into a course on Friday in a new discipline (psychology), with an unknown professor, into a post-positivist world when I have been firmly living in social constructs for the past year, and I may really have no hot clue how well I’ll perform or live up to expectations. Writing self-efficacy may not be able to adequately predict grades. It may however predict the behaviours you require to get a good grade. It may also predict your willingness to keep writing. The only thing that will make your writing grades better is more writing. And are grades really a good reflection of the quality of a writing product anyway? Food for thought.
  3. I believe that the way in which people cognitively interact with a questionnaire and come to a decision on what score to give themselves is a complex process. And this is one part of my thesis project that has evolved dramatically. I was going to do straight up psychometrics — factor analysis, multivariate statistics — but I want to know more than that. So I will develop the questionnaire based on my constructivist theory and I will do think-aloud interviews with students to assess how they interpret the items and come to a decision on how to score themselves. Cognitive interviewing, the psychologists call it. So the project has become more qualitative. I will also use a delphi panel to help me with final edits. The question is, what comes first, the delphi or the think-aloud interviews…. Hmmm.
  4. I’m becoming more interdisciplinary in my thinking. Strange since I’ve been immersed in the nursing world for all my courses and my teaching but what I am doing is not just for nursing. I’m discovering quickly that my work will spread further if I quit spinning it for nursing journals. I published the questionnaire paper and it was really good. The theory paper is awesome and I called it a theory for nursing education but…… it is a theory for all disciplines. It’s almost too bad that I sent it to a nursing journal but I also had some bones to pick with nursing and their writing publications so it is OK there. I’ve published a few other studies that have had some interesting findings and I’ve had more than one moment of being ready to lose my shit with some of the overly structured rules attached to some nursing journals. I nearly pulled one submission recently because of that. I had a great journal choice in educational psychology all picked out as my target for resubmitting and then when I went to read the paper for fit, it was all nursing this, nursing that.…… and it was going to be more work to remove the nursing spin than I was willing to do. I just want the damn paper published. I fear that the psychology people doing work in writing self-efficacy won’t find my papers in order to cite them. They will be unlikely to search CINAHL for this topic — for good reason.
  5. My study needs to be about more than about undergraduate writing. I was going to only interview undergrads but the fact of the matter is that I do want the questionnaire to be applicable to research on grad students as well. I also don’t want the questionnaire to be only applicable to nursing education. It needs to be interdisciplinary.

I need to be thinking about writing my research proposal soon even though I am about a year away from being ready to move to that stage. I’ve written now 4-5 papers that have required me to summarize and present a review of the literature on writing self-efficacy. It is going to be tough to find yet another way to write about the same findings without self-plagairising.

I still have a lot of reading left to do. The pile in the photo is all the articles that I have collected since summer of things I want to read. Some of those articles are about construct validity in writing and assessing writing outcomes so I hope to fit them into my  work this term. Hence, since I often focus this blog, and my Twitter on what I am currently focusing on, there may be a little bit of a flavour change in what I write about for the next three months as I explore measurement, and hopefully, measurement as it relates to writing.

 

*In some ways, I would love to change my topic. I have been introduced to all kinds of shiny things that have grabbed my passions — eg. Narrative Inquiry, for one. But I have a committee now set up to get me through a measurement project so I carry on. And, this IS the next step in my work, this tool development. The big qualitative study will come after.

Do Academics Devalue Writing?

Posted on by academicswrite

I meant to write this blog months ago. It is inspired by observations made while at a conference in the spring with other writing scholars. The devaluing of writing is an issue I think about often as I inch ahead in my own PhD studies and consider, while applying for grants for example, how others will perceive my work. How my discipline perceives my work is especially crucial given that I am in a practice-based discipline not a humanities discipline where, you would think but it is apparently not so, that the value of writing would be more self-evident.

My research area is writing in nursing education.  I have conducted research on writing self-efficacy in first-year nursing students in several projects I started before I became a PhD student. When tackling psycho-educational topics in a discipline that privileges the biomedical perspective, there are those that will brush you off, tell you your topic doesn’t interest them, or look at you strangely and tell you they didn’t realize nurses needed to write. In that environment, it is hard to go up against, for grants or even to get certain journals to take you seriously, scholars who are trying to cure cancer, as the epitomes example.

But someone has to teach the future cancer curers of this academic world how to write and think.

In nursing the problem has looked like such:

  1. Most nursing programs in Canada do not teach writing at all to their students. Or if they do believe their students require writing skills, they require an English Literature course as a prerequisite (as if interpreting literature and writing about it will help them write better research synthesis papers). Or they require students take a basic generic writing course which drills grammar and a style guide. Discipline-specific writing instruction is rare in nursing programs (6% in Canada).  I am grateful to Jo-anne Andre and Roger Graves for the study that has fed me that stat.
  2. Writing ability is not an entry level practice competency for nurses at the national or provincial level in Canada.  I have been through the competency documents for every province (except Quebec) and none of them include writing as a competency. One of them (my own province, gratefully) includes the following statement under the heading of assumptions: “Entry-level registered nurses demonstrate English or French language proficiency (reading, writing, listening and speaking).”
  3. The Canadian Association of Schools of Nursing (CASN) the organization that “establishes and promotes national standards of excellence for nursing education”  has written a National Nursing Education Framework. Writing does not appear in the framework as a priority of nursing education until the Masters level of education. Writing is not mentioned in undergraduate education at all so I’ve been wondering where nursing Masters students are learning to develop such skills as as,
    “The ability to articulate verbally, and in writing, to a wide range of audiences the evidence for nursing decisions, including the credibility and relevance of sources of information,”

    if those skills are not honed in undergraduate education as a component of excellence in nursing education. The hardest writing requirement imbedded in that statement above as an educational priority for Masters education is the writing for a wide range of audiences. Understanding the needs of an audience is the most challenging writing skill of all. You come to understand the needs of an audience, through lots and lots and lots of writing and that writing exposure better come well before the Masters level.

I’ve been fortunate enough to be on faculty in a nursing program that includes in its curriculum one of the rare 6% of discipline-specific writing courses. In fact, I developed that course. Our former chair, now retired, valued writing, but it is likely she included the course because she had a ready-made registered nurse faculty member (meaning me) to step in and develop the course. Our course, in its inception, was truly discipline-specific because it was taught and developed by a registered nurse. But she only made it one credit hour in value which does not represent the workload or the stress levels incurred by some students.

Nevertheless, I’ve still encountered problems from our faculty over the value of writing. For example, we have a parallel policy that students must pass every course in each term before moving on in the program. We weren’t far into our new curriculum (2 terms) when the writing course became a quick exception to that rule and students were were allowed to move to the next term without successfully completing the writing course.

As another example, at some point in our program we had to stop allowing faculty to be too autonomous in their assignment choices and the location of writing assignments needed to be pre-selected and permanent because writing assignments were being dropped from courses without any consultation. Instructors would trial an assignment, quickly realize how much work they were, and the next year it would be gone from their course. They did this stealthily, without telling anyone, and no one found out until it was too late.

I have also discovered how quick instructors are to place blame on the introductory writing course as the cause of students’ perceived lack of writing skills. I can’t even count the number of emails, hallway conversations, and pointed questions at faculty meetings, where I was required to address the generic finger point, “Didn’t they learn this stuff in your course?” As if I was the magic bullet. As if my course was the end of the line for undergraduate students learning how to write.

So it shocked me to be at a writing scholars’ conference (CASDW) and find out that the nursing experience wasn’t unique at all. That even in the writing studies discipline faculty were plagued with pointed fingers and statements of devaluing. Faculty calling writing a “soft skill.” The blaming of some unknown entity before these students arrived at their doorstep for their poor writing as if each individual course assignment didn’t require new learning, new writing supports, no matter the level these students were at. We seem to want students to be sitting in our classrooms fully formed when it comes to writing. And the industries we feed want students coming out with better writing (and communication) skills but they don’t want to lengthen programs to help students develop these skills.

When I wrote the tweet I posted above, many of those who contributed to the conversation thought I meant students were the ones devaluing writing but I was talking about faculty primarily. Many defined students’ devaluing of writing by them not caring about the grammar in their assignments. But grammar is not the only thing that makes bad writing. And what makes for bad grammar can be highly subjective and disciplinary too. What bothers me as a grader will be different from what bothers you. I, for example, could not care less about detecting split infinitives or sentences ending in prepositions, but I’m going to be all over bad uses of semi-colons. I’m much more interested in ideas, clarity, creativity, cohesiveness.

Reading qualitative studies asking students about what they think of the feedback they get on their writing enlightened me to student frustrations. The thing students hate the most is when their graders fail to see what they are trying to say in order to simply nit-pick at sentence by sentence grammatical structure. You want students to pay attention to grammar, tell them to read their papers out loud to themselves. Give them time in class to do it. More writing will improve students grammar but students quickly become disengaged in their writing if they feel their ideas are being ignored.

If students devalue writing it is because we model that to devalue it is acceptable. We model it by doing some of the things I’ve described above.  Allowing them to progress in a program without passing a required writing credit is like saying, well, you can get by without it. I’ve heard faculty talking to students about how much they hate writing too and avoid it. I’ve heard them validating student beliefs that nurses don’t need to know how to write to look after a patient. Faculty make these comments without any consideration for the nature of thinking that goes into writing that will benefit student thinking at the bedside.

In my experience, the faculty complaining loudest about the dire condition of student writing are the ones that seem to devalue writing the most. Many of these faculty have no intention of being a part of the solution. Many don’t recognize that in order for good writing to be handed in, supports must be in place and the educator assigning the writing must be a part of that support system. Bad writing in your course is not someone else’s problem. It is your problem. Writing experts have known for years that drilling grammar does not fix that problem, so demanding that writing scholars come in and fix the issue by offering a 2 hour workshop on the basics of grammar, will not fix the bad writing your assignment produces. You’ll be lucky if any of the students show up. Deficit pedagogy, where we tell students what NOT to do over and over again, does nothing to teach them what they should do.

In  my mind, getting good writing out of our students requires three simple things:

  1. Writing a meaningful assignment and allowing student choice.
  2. Providing in-classroom supports for our own assignments.
  3. Allowing students to say something that is their own and represents their identity in the work.

Developing writing identity may be the key to helping students value writing. Students resent writing that demands they leave themselves out of the analysis. I don’t blame them. I resent writing like that as well. But so many disciplines continue to devalue writing, even at the professional academic level, that shows any shred of humanity.  They label that kind of writing as biased writing, lacking objectivity. I conducted a poll shortly after tweeting the devaluing thread, asking academics and researchers if they would call themselves “a writer”

If you remove the folks that were just spying, 56% said yes, 30% said no, and 14% said not sure. So just under half of the academics/researchers and Phd students who responded would not identify as a writer. In the comments below, some said they they felt writing was a necessary evil of the job. It was a task, not an identity.  Some felt it was not their primary identity (teaching was). But yet writing is what we do. Writing is what makes our research travel. Writing is what gets us degrees, promotions, grants, recognition, publications, and advances our careers. How can we not identify as writers?

If such a large proportion of those teaching and assigning writing to students cannot identify as writers then we have an identity crisis in academia. The problem of devaluing writing may stem from this identity crisis. I learned this week that writing studies scholars have challenges even being recognized as a relevant discipline. If we don’t write as academics, if we don’t value writing ourselves, if we don’t want to teach writing or help our students value writing, what is it that we do again?