The Consistency Conundrum Part II: How to build Flexibility into Writing Instruction

Posted on by academicswrite

We learned in Part I of this blog how attempting to fix apparent lack of consistency in writing instruction and grading through rigid practices of standardization, actually had the opposite effect and seemed to increased perceptions of inconsistency. Standardization also caused students to disengage with writing and created inequitable circumstances for minority and other disadvantaged student populations. So how can we build flexibility into writing instructional practices and reduce perceptions of inconsistency?

What does flexibility benefit? 

Flexibility allows for adjustments in requirements and even expectations from student to student depending on topic choice needs or support needs. The student with a difficult to research topic who has exhausted all avenues to find that 5th required source, but is so passionate about the topic they don’t want to abandon it, will be provided with an alternative. The student who did not have a clinical experience that is a good match for the assignment will be allowed to draw an experience from another domain.  

Flexibility also takes student focus off the question of, What does my teacher want me to say exactly so I can get an A, and redirects focus to the writing act itself. 

Some contexts require more structure than others.  You still have to create an assignment that helps the students learn something critical to their discipline or the substantive area of your course, whether that be how to argue, define a threshold concepts, or how to compare and contrast theories. If one of the objectives in your course is teaching students how to be concise, then stricter rules on word count and other format issues might be necessary. Students new to university might need more structure than students who are closer to graduation.  Loosening the reins does not mean no structure at all, in most cases. But, the caveat is, if you choose structure in some part of your assignment it needs to serve student learning or be an element of scaffolding.

Disclaimer: Writing is disciplinary. My experience is nursing education so my examples necessarily fit what I’ve witnessed in nursing education. You may have to make some metacognitive leaps for how things work in your own context.

The five assignment components that are the most frequent victims of standardization

I see five main parts to assignment structure and from within those 5 key areas you pick and choose, like an à la carte menu, where the structure will be most helpful to guide students through the assignment. 

  1. The framework for the writing, thesis, or guidelines
  2. The content topic of writing.
  3. The process and/or research requirements
  4. The rubric
  5. The mechanical format

Wherever possible, choose a flexible option. Ask yourself, How much scaffolding does your particular class need? Fexibility-Inflexibility is also not an absolute dichotomy.  Often the line between flexible and inflexible is also about faculty mindset. You can ask for structure but hold a flexible mindset in how that structure is applied. You can demand the student use 5 primary studies to explore research results on a topic (this is almost always possible to achieve in any topic) but be open-minded because for certain topics, the perfect studies may not exist. Negotiate with the student who’s topics do not neatly fit your prescribed structure. 

Be honest with yourself about your ability to hold a flexible mindset. Give your class the benefits of the doubt in terms of what their capabilities are. Always, always assume they are capable of more than you think they are. Be prepared to do the work to get them to that level of capability. Be mindful. Choosing more structure over less, often benefits the teacher and their control needs more than it benefits the students. 

So, let’s talk about what might be flexible vs. inflexible options among these 5 key areas.

The framework for the writing, thesis, or guidelines

Inflexible: Providing students with the exact headings they need to use and the detail of exactly what should go under each heading. Or providing a theoretical framework that students should follow to structure their writing. There are varying degrees of structure here as well, from providing headings that are more akin to pointing out genre form (such as IMRAD — introduction, method, results, and discussion) that they may not be able to recognize on their own. IMRAD or traditional review of the literature ordering (e.g. background, source inclusion, themes from the literature) is a structure, but one with much room for creativity. But if you are pointing out specific detailed content expectations or specific questions that need to be answered, you may be giving too much structure. Most of the time, what goes under the headings will be specific to the student’s chosen topic. 

Flexible: Providing a general prompt that allows for student development of their own organizational structure for the writing that tells the story of their topic in a way that makes sense to them. Let them find their own applicable frameworks to guide their writing based on their own explorations of a topic.

The content topic of writing

Inflexible: Assigning every student the exact same content topic or providing a short list of limited topic choices. This might also include providing the same case study for every student to examine within the thesis of the paper or a limited set of case studies. It is a common practice in literature courses to give the same content topic to every student (the theme of eating in Great Expectations), but usually the flexibility comes in under variations for process approach or research requirements. 

Flexible: Allowing students to choose their own topic or develop their own case study. 

The process/and or researching requirements 

Inflexible: Providing students with the sources they must write from and limit them to those sources.  

Flexible: Allowing students to find their own sources. This can still mean they can be given guidelines for the numbers and types of sources they use, which is a little less flexible but ensures that they’ll use quality sources. Another flexibility alternative might be allowing students to discover their own process for researching the paper or gathering the information they need to write the paper, which might mean interviewing individuals or creating surveys, conducing environmental scans or field work, depending on the requirements of the assignment. 

The rubric

Inflexible: Constructing a content heavy rubric which lists specific content expectations such as, “must talk about 5 different ethical values in relation to the case study.” Or, your entire department is using the exact same rubric for every “academic writing” essay assignment. 

Flexible: Developing quality indicators of a successful essay to assess how content was addressed, the depth of critical analysis, the quality of the research done and synthesized into the writing. This is hard to describe as every essay will require rethinking how to describe the markers of high-quality writing and content presentation as applicable to that assignment’s needs. 

The mechanical format

Inflexible: Formatting demands in assignment guidelines are really managed through mindset. Requesting a specific number of paragraphs with a specific number of sentences in a particular order will cause students to focus more on structure than flow of writing. Mindset inflexibility includes grading approaches that require black and white correct implementation of a style guide format that allows for no errors to grammar or the style guide implementation; or, grading discourse errors as if they are grammar errors and rewriting student sentences. Heavily penalizing mechanical things defeats the purpose of using writing as a mechanism for learning and identity development and students in dozens of studies have talked about how they resent having their ideas ignored. And for you grammar pundits, I am in no way saying you should not be grading grammar or mechanics at all — just be sure you aren’t ignoring the other stuff.

Flexible: Providing general formatting requirements such as an overall page length or word count. Most of the time it serves no one (but the teacher) to be too demanding about sticking to those page guidelines. I always judged too short or too long papers on how well they addressed the assignment guidelines with depth and completeness or through writing about extraneous things. Page limits were just guidelines – but well-chosen page limits are based on experience and realism. Choose page limits wisely. I’ve seen assignments that requested tight page limits, students were told graders would stop reading at that page limit, and then the feedback pointed out numerous areas were students lacked depth. A student should be able to write an assignment with the appropriate degree of depth that matches your expectations within the given page limit. Style guide flexibility means expecting consistent implementation of a style guide but recognizing it is near impossible to be mistake free. Grammar flexibility means recognizing that grammar is subjective so one should avoid overly penalizing for errors that might be your personal pet peeves. Be aware of of how grammar evolves over time and how previously unacceptable practices can become acceptable (does anyone know what a split infinitive is anymore?). Allow students to have their own voice. Correcting discourse errors but don’t penalize through the grading process. 

Here is an example of a discourse edit where there is nothing grammatically wrong with the first sentence but the way it is said is just “sounds funny” for the nursing discipline:

Student Original: This paper will focus on five qualitative articles that discuss the experiences of adults with epilepsy seizure disease and how they can adjust themselves to have quality of life

Corrected at Grading: This paper will focus on five qualitative articles that discuss the experiences of adults with epileptic seizure disease and how they can achieve a sense of quality of life

It’s very difficult for me to visualize every possible assignment that exists in higher education. Much of what I wrote above is disciplinary and may fit my discipline better than yours. I hope the basics are clear here and you can see how to adjust for your own context.  If you’d like to see a specific example of mixing and matching structure and flexibility, I direct you to this blog that describes one of my own assignments. It is flexible in content topic choice, rubric style, formatting, and research/process direction (with specific direction on resource source types because at the level my students are at, they need guidance with that). It is inflexible in its framework/guidelines but on the level of providing genre-based headings to follow– my students needed lots of direction in defining what some research concepts meant, especially the concept of “practice implications,” so they could correctly address the content needed under that heading.

Like with the nursing skills example in part I of this blog where I’m sure we all believed we were being consistent, it can be fairly easy to THINK you are being flexible when your students might feel otherwise.  Be self-reflective on that as you apply this framework to evaluating your own assignments. Pay attention to what they ask for when they ask if they can make modifications to your assignment. That might be signal as to them feeling their ideas do not quite fit the mold of your assignment.

How do you mix and match flexibility and structure in your writing assignments?

The Consistency Conundrum Part I: Is Inconsistency in Writing Instruction Really a Problem?

Posted on by academicswrite

Many many years ago when I was a new nursing instructor and I was getting assigned to the things that nursing instructors get assigned to do, I was, for a few years, plunked in the skills lab for our spring session course.

We got the same instructions every orientation from the instructors assigned to organize the course. “Every person in this room, has to teach the skill in exactly the same way.” 

And we’d all nod our heads and go off and teach it the way we knew how to do the skill. I guarantee we all THOUGHT we were teaching it the same way because, it’s a skill. It’s objective. There is only one way to do it, right? 

My way. The right way. Which is the same as your way. Of course. 

Why were we demanding skills consistency? I don’t remember any reason beyond the fact that it was students’ number one complaint on their course evaluations. What they were taught seemed to them to be very different from teacher to teacher. 

We weren’t consistent. 

Nearly 15 years later, we haven’t solved that problem. They still complain of inconsistency in skills instruction.

We never really got specific information about what part of the psychomotor activity wasn’t consistent from teacher to teacher, because psychomotor skills in nursing are really a set of safety and infection control principles that need to be adjusted slightly for context, but it seemed their consistency issues were as nuanced as hand position when holding the forceps that were holding the sterile dressing; or — here is a timely one — do you put your mask on before you put on your sterile gown and gloves last, or some other order? And does it actually matter? How drastically different could our demonstrations be from one teacher to another? 

Why do we worry about consistency at all?

I began to question, why then, if it was not possible for instructors to present a guise of consistency with a psychomotor skill, why did we worry about consistency at all? What prompted complaints of inconsistency? And wasn’t consistency just about different teaching styles? The inconsistency problem forced me to sit through countless overly long skills demonstrations by our course leader attempting to get us to all agree on the procedure.

What resulted was 8 instructors interrupting her, Well, at my hospital, we do it this way. 

I hold a pretty constant stance of being norm-critical, even back in those early days of my teaching career. I can remember, in an attempt to put myself out of the constant misery of the consistency meetings, speaking up and saying something like, “Perhaps, we should instead teach students to embrace the ambiguity. They need to learn to be adaptable and recognize when principles are being applied instead of obsessing over minor differences in technical performance.”  

I think in some circumstances, inconsistency can be a real problem – as I complain about it incessantly every time I submit an expense claim at my institution and the rules are different, yet again. I can understand why in a lab experiment, you have to use the exact same number and drops of chemical using the same dropper in each replication. But in the real world we are dealing with independent thinking human beings. For the most part, what we and students call inconsistency is really just a euphemistic way to say, I feel things are unfair. It is probably a genuine lack of fairness — as recent events have taught us, inequities are perpetual in higher education.

That lack of fairness, however, in my opinion, is born out of insistent adherence to rigidity and black and white thinking in a context that is nuanced with situations that are never identical from person to person. And of course, rules always favour the privileged.

And here is a key point: students don’t complain about inconsistency unless they feel it has caused them personally to lose marks or be evaluated differently in some way. 

Transferring consistency practices to writing instruction

Now let’s talk about what I am really here to talk about – consistency in writing instruction and grading. I mean, yeah, writing is not a psychomotor skill.  This set of blog posts is inspired by a Twitter thread I wrote on the topic on the weekend. Then the US election was called and the thread sort of sunk into oblivion, but I feel I had more to say on the subject. It also ties in nicely with my PhD thesis work, so, hence, I blog.  

You see, I believe the problem with psychomotor skill inconsistency wasn’t really with inability to tolerate minor differences in skill performance between instructors, it was more to do with inability to tolerate perceived differences in how they were evaluated on those skills. It was about grading. It was about feeling criticized when a teacher made a suggestion to change a hand position or do something in a different order which was not EXACTLY how they’d been taught. Those suggestions might have been about efficiency rather than evaluation, but – and this is a huge problem in clinical education – it’s really hard for a student to tell the difference between evaluation and just a suggestion when they always feel like they are being evaluated.  

They have the same complaints about writing. They could tell me about all the things in their writing lives they felt were unfair.

“I had to use her words, her way, her everything,” one student told me during the interview phase of my research. “Otherwise I got a bad grade.”  

Another student told me about getting a bad grade because they didn’t have a clinical experience that particular term that fit neatly into the assignment guideline instructions. This student asked the teacher if they could use an alternative — an experience from a previous term. The teacher refused to allow adjustments. The student, then, could only artificially address some of the required aspects of the writing assignment. No surprise, assignment comes back with a reduced grade.

These examples are of teacher behaviours that overgeneralize perceived rules that don’t take context into consideration. The first student felt like her writing voice was not valued. The second student didn’t understand why an alternative experience that was a better fit for the assignment, couldn’t be used. What did it matter what clinical rotation the experience was drawn from?

And both students could tell me stories of the opposite happening to them or other students with the same or different teachers — flexible teachers — which is what makes observing differences in instructor responses feel all the more unjust when you are the student forced to artificially fit your work into some tight boundary. There was no consistency.

What happens when we attempt to fix inconsistency with standardization?

So, what tends to happen when we attempt to respond to student complains of inconsistency in our programs?  We standardize things. We standardize things because we think its like having to use the same dropper in all replications of a chemical experiment — it is more rigourous. Sometimes we implement standardization practices on our own and sometimes it is handed down to us from administration – “everyone will use this standardized rubric for academic papers.” We give more instructions instead of less. We remove choice. We prescribe the exact details of what we expect students to say in their papers to the point that the guidelines are longer than what students are expected to hand in.  

In my doctoral research, students described a really rigid writing environment. They told me that writing in nursing was actually “pretty basic,” “fill-in-the-blank,” everything was right there in the rubric – put this, this, and this, for 5 points, no wiggle room for creativity. I’m loosely typing their exact words. All 20 students in my study described some kind of rigid context for writing in at least one nursing classroom which suggested to me that a push toward standardization is the norm rather than the exception.

And, guess what — this won’t be a shock — they still complained of inconsistency. You can read more about what my students said about the writing context specific to nursing education in my previous blog: It depends on the professor. 

Faculty in the two nursing programs I sampled from in my study, turned to what they thought to be an objectivist solution – standardization, very specific rubrics, identical fill-in-the blank type writing assignments – and it did not solve the problem of inconsistency.  Consistency is really another word for reliability. The lack of reliability in grading was both inter- and intra-rater. The perception of inconsistency existed from student to student whether those two students were being graded by the same teacher or different teachers. 

Herein lies the paradox: In addition to still complaining about inconsistency, some students also resented being hemmed in. They resented that they couldn’t write about the things they were interested in. It made them put less effort into writing and researching – “it will be the first sources I find…. BOOM…. That’s what I’ll use to write my paper.”  (Again, loosely quoted, except for the BOOM).

So standardization kills exploration, curiosity, creativity, innovation, independent thinking. It also trains students to expect you to deliver assignments that provide them with 100% certainty and eliminates all the ambiguity from the learning environment. Is that a worthy trade off? Especially when that standardization has essentially no effect on the problem it was designed to solve in the first place… inconsistency.  

Like using the same dropper in a chemistry experiment, standardization attempts to set up equal conditions for all students. It professes to increase fairness. But we know students aren’t all standing on equal ground. My data showed that privileged students – students with previous educational experience or who had family members who understood higher education – had no problems adjusting from teacher to teacher and their different rules, pet peeves, and biases. They had the skills and knowledge of academic life, to do that and accepted it as normative. But international students, Indigenous students, and students from first-gen and disadvantaged backgrounds felt they could never figure it out their teacher expectations. To them, the so called rigid rules were a bunch of senseless arbitrariness. 

Standardization, privileges the already privileged.  

Why do we keep thinking standardization works? 

I sense we are uncomfortable with our students’ discomfort so we are driven to eliminate it, even though learning is not supposed to be all comfort and roses. I also think standardization is implemented to serve more than just the reliability problem. It also is implemented to quell the anxieties among faculty who are not comfortable assigning and grading writing.  Maybe they feel assigning the same fill-in-the-blank style “essay” (I use that word very lightly here) on the same topic to every student will help them better compare one student to another and more fairly assign grades? I’ve also heard the argument from some faculty that they have to control the topics students choose because they don’t want students choosing a topic they don’t know much about themselves because they won’t know if the student addressed the topic correctly. 

But all of this is anecdotal. There is not much research exploring what drives faculty decisions in creating writing assignments in nursing…. Or in other disciplines for that matter. 

Needing to control the content is a sign of anxiety as well. It’s like faculty doubt their own skills to be discerning readers who are capable of recognizing clarity in text and engaging writing. Faculty, as consumers of academic material, have the ability to know when they are learning something, when ideas are presented effectively, and when something seems off or improbable in the information given. 

Anxieties like this are not uncommon among faculty in nursing programs. Most nursing instructors are not PhD prepared. Nursing has not yet reached that state of PhD prepared surpluses present in most other disciplines. Most individuals teaching undergraduate courses in nursing are Master’s prepared. They also tend to be more comfortable in the nursing practice area rather than the academic arena. We need to better understand their learning needs for writing instruction. 

Flexibility breeds perceptions of consistency 

I think we’ve been thinking about the inconsistency issue all wrong. Structuring and standardization makes complaints of inconsistency increase through providing a false sense of objectivism. Standardization isn’t really standardization at all when everyone reads the so-called standard and develops a different interpretation for defining what is a correct representation of that standard. 

Standardization then removes validity from the assignment; thus the assignment no longer elicits the learning it was designed to assess. Students are so focused on rules and APA and grades that they no longer think about content or engaging a reader. The reader is the teacher. We instead assess ability to follow rules and deliver what a teacher wants. Students in their course evaluations then question fairness and question the point of writing. The students in my research who could not figure out how to read teachers, resented being hemmed in, and they disengaged with writing.

I’ve often said, we are key contributors to student bad writing by making them write things they don’t care about. How then can flexibility help?

Next: Part II. Looking at using flexible practices to reduce inconsistency.

Also: Watch the video that outlines some of my study findings.

Navigating Undergraduate Academic Writing: Guess What? It Depends on the Professor

Posted on by academicswrite

It happened in one of my early student interviews. The student read the item on my developing questionnaire that asked them to self-assess their ability to adapt their writing to meet the needs of the assignment. The question was supposed to be getting at switching from genre to genre. If they were writing a reflection, did they sound reflective? If they wrote an academic assignment, did they get their point across in the form of an argument and address a specific question? But her response was, “I’m meeting the needs of the professor, not necessarily meeting the needs of the assignment.”

And it kept happening over and over – sometimes with the same question sometimes with other questions – the students in my sample talked about needing to figure out the teacher. It wasn’t just a handful of the students in the sample. It was 100% of the students in my sample. They pondered issues such as: What were her preferences? What biases does she have? Is she an easy or hard grader? How approachable is she? How open is she to re-interpreting assignment guidelines with uncertain students? How well does this teacher actually understand academic writing themselves?

There is a novice to expert continuum with guiding writing as there is a novice to expert continuum with becoming a writer. Students can tell when faculty are not confident in their own skills to assign and evaluate writing – and this state is not uncommon in undergraduate nursing which can be taught by faculty who aren’t required to write and publish themselves. Novice instructors are far more likely to hold tight reigns on their assignments and rubrics and be overly focused on surface textual features.

Navigating the writing context is like a journey down the yellow brick road in search of the great and powerful Oz (the teacher) who can solve all their academic writing needs. When they get to know Oz, they sometimes find Oz is not the all-powerful being that they’ve hoped to find. Oz is flawed and fallible, but Oz does have the power to help students recognize that the true power of being a good writer was within them all along.

Students had various strategies for success in this mission of “figuring out” the teacher such as: paying attention to a teacher’s overt opinions on a subject and matching them; noticing what words teachers like to use and slipping them into their own writing;  talking to their classmates on how they saw the assignment; talking to classmates who had already been through the class to see how they perceived the assignment and the teacher; testing out ideas about topics with instructors and peers; judging their relationship with the teacher to see if she can “take it.”

Other students seemed to never be able to figure out teachers no matter what they did. Or they felt they could learn to read the teacher if they ever got the opportunity to write for them more than once in a term — which rarely happened.  Their biggest faux pas in this area seemed to be that many of these students would never actually approach a teacher to ask questions, or they felt that when they did, their questions were misunderstood or brushed aside. Often these students were the ones that you would most want to come and speak to you – immigrant students, students where they learned English at an older age such as when a teen or an adult, and, in nursing, it was also often male students.  These are our students who perhaps struggle to feel like they belong because they don’t fit the stereotype of who a nurse is – passive, polite, white females. And for those who may be thinking, It isn’t like that anymore — implicitly these traits in nursing are still the most highly rewarded and have an easier journey navigating the nursing world.

The opinionated students don’t fit in well either as they often insist on maintaining their own opinion and making things their own. They are the ones that twist assignment criteria to ensure they can say what they want to say. Sometimes these behaviours are highly rewarded, but other times it means they fail the assignment.

It depends on the professor.

What concerns me most is what this means for writing as a pedagogy. If the strongest students are graded highly because they are able to spit back to the teacher exactly what they want to hear, if the writing that is most highly rewarded is the writing that fits into a nice little box where the number of checkmarks on a rubric is the only indicator of learning and quality, we have a problem. It means higher education is not about independent thinking. It’s not about recognizing where boundaries can be stretched and alternative perspectives are valued.

The undergraduate students in my sample spoke about a very problematic culture of writing in higher education. I will speak broadly to this because while I think nursing has some work to do, I don’t think some of the problems I see in the reports I heard from nursing students specifically are exclusive to nursing contexts. Students spoke about “bullet-point” assignments, “fill-in-the-blank” style assignment guidelines. A few students – and this especially came from students who had experience as students in other disciplines such as psychology, English, or gender studies – recognized that the writing was different in nursing.  It was more “simple.”  It was certainly often inflexible. You must “stay within script”. You can be creative but only “within the dimensions they want us to be.” There are boundaries. Being creative, depending on the professor, could mean losing 10% of your grade. There are far too many assignments where there isn’t enough choice and students are forced to write about topics they do not care about. For those students, in those moments, writing becomes a functional and robotic task.

Why do we not have enough faith in our undergraduates to be able to give them free reign to go where their ideas and passions take them?

Are we afraid that we can’t fairly grade if our 60 students aren’t handing in 60 cookie cutter assignments all answering the exact same question using the exact same literature? Who wants to grade those? We lament grading because of the things we do to ourselves in assignment guidelines. I’m asleep already just thinking about it.

Are we afraid if we don’t know the topic to the same degree an expert would know the topic that we aren’t capable of accurately assessing writing?  I don’t know about that – I know when someone has written something in a way that is accessible to me and when they’ve caught my attention and I learned something. Without knowing much about the topic, I am also capable of being sceptical when the accuracy seems “off” or there seems there is another side of the story that isn’t being presented. I am an astute reader. I assume all faculty are astute readers too.

Do we tighten our rubrics into minute details because then we are able to be more reliable in our grading across students or between raters?  But what are we sacrificing in terms of assignment validity in the process? Are we still able to capture the objectives we are hoping our students leave our classroom with at the end of the term? Seriously, just feed them a multiple-choice test if you want strict criteria and one correct answer.

I’m getting a little ranty. I can feel it.  But this is a hill I am willing to die on.  If you are not giving writing assignments that students can get excited about – and I mean every student – your students are learning you, not the threshold concepts of your subject matter. Students are equating their success at writing with their ability to be successful in navigating the writing context, teacher preference, and their levels of writing self-efficacy are impacted as a result. Papers are not about one correct answer. Writing is not about the text produced, I mean, yeah, you want it to be readable and not “messy” – students get that.  As one of my student participants said, “It almost becomes part of your identity. Like you care more about that subject when you go to other places. And then you’re also looking into the literature, so what do other people know? And you’re sort of building yourself and your knowledge base.”  A student walking away from your course should feel that about every assignment.

This little rant is courtesy of a preliminary analysis emerging from a secondary analysis of Congitive Interviews conducted with the primary objective of editing the developing Situated Academic Writing Self-Efficacy Scale.