If Doctoral programs didn’t change your thinking, they wouldn’t be doing their job would they? Here at the start of a new year, I thought I might reflect upon what has happened to my thinking on my planned thesis project to develop a measurement instrument to assess writing self-efficacy.
I finished all my required course work toward my doctoral degree last month and I’m itching for the next steps. I still have one more course to take and that is an elective I, and my committee, have agreed upon which will fine-tune my skills in measurement of psychological concepts and the statistics of assessing those measurements. I’m really looking forward to the change in pace as I have been immersed in philosophical ramblings for quite some time now and that is hard thinking. Something a little more “rule based” and structured might be nice. I say that now but I’ll be frustrated, no doubt, by the particulars in no time. In some ways, taking the course is a bit for show on my transcript so no one questions where I got my measurement training from when I go to defend. I would rather sit and read a hundred articles on my own and figure it out with textbooks and conversations. The bad thing about courses is that the structure I just admitted to craving, hems you in. I really hope I have some flexibility in terms of what I read about and how I tackle my assignments but that is usually not the case.
Since 2011 I’ve been studying writing self-efficacy. I’d like to say I fell into that area of research inspired by something profound I read or a conversation I had but it was quite happenstance and to some degree arbitrary. I had read nothing. I just knew my students lacked self-efficacy about their abilities to write the paper I assigned them. I’ve since read a lot and my thinking has shaped — it is a little less a big lump of clay… it’s taking form. I have opinions. I am developing expertise.
Before I even entered my PhD program I had conducted three studies and a questionnaire review on the topic. I knew when I was writing my please-admit-me letter that I wanted to develop a measurement instrument to measure writing self-efficacy. Nothing about that has changed. I’m going forth. But my thinking about how to approach the project has changed a lot. One of my classmates just asked me recently how it is I’ve managed to get this far and not change my topic.* (She, incidentally, has changed her study focus three times). My response was, first, that it was a bit of pragmatism…. the most direct route to graduation so I could get on with doing exciting and meaningful stuff.
My second response was that it had changed, philosophically. I wouldn’t have considered myself a theoretical thinker when I wrote that admission letter — that turns out to be absolutely not true, and slightly lacking in self-awareness. The originall vision was straight up statistical psychometrics. But, partially because I had to for a course, I developed a constructivist model of writing (for nursing) — bracketed for a reason — which I revised and sent back to a journal at their request over the holidays. But the reading for that has lead to other thoughts about writing self-efficacy, my chosen concept. I chose the concept when I had read nothing but now I have read plenty.
- Constructivism is the road to better measurement of writing self-efficacy. Writing has been through three epistemological shifts (product, process, social) that happened in fairly rapid succession and the tools that measure writing self-efficacy reflect that. The earliest tools assessed it by grammar fault and ability to construct sentences and be clear. Later ones took a more cognitive process, motivational, self-regulation, perspective. But none of the tools take a social constructivist perspective. Some of the tools have the occasional item that brushes up against constructivism but they don’t capture all the social aspects of writing bound to affect writing self-efficacy. How do I know this? I did the work and it was published in the Journal of Nursing Measurement along with an accompanying editorial.
- Writing self-efficacy does not have as strong of a relationship to grades as we would like to think. I certainly have not seen any convincing evidence in my own studies or anyone else, that it actually predicts grades…. at least not in a real-world relevant way. (In health research they would call what I am talking about clinical significance.) Part of this prediction failure is related to context. People assess their self-efficacy based on previous performance but in the face of a new teacher, a new subject, a new discipline, new rules, they may assess their own ability poorly. I for example would tell you right now that I believe I have fairly strong skills and knowledge of measurement based on the reading I’ve done and my research experience. I should ace my measurement course without difficulty. But I’m walking into a course on Friday in a new discipline (psychology), with an unknown professor, into a post-positivist world when I have been firmly living in social constructs for the past year, and I may really have no hot clue how well I’ll perform or live up to expectations. Writing self-efficacy may not be able to adequately predict grades. It may however predict the behaviours you require to get a good grade. It may also predict your willingness to keep writing. The only thing that will make your writing grades better is more writing. And are grades really a good reflection of the quality of a writing product anyway? Food for thought.
- I believe that the way in which people cognitively interact with a questionnaire and come to a decision on what score to give themselves is a complex process. And this is one part of my thesis project that has evolved dramatically. I was going to do straight up psychometrics — factor analysis, multivariate statistics — but I want to know more than that. So I will develop the questionnaire based on my constructivist theory and I will do think-aloud interviews with students to assess how they interpret the items and come to a decision on how to score themselves. Cognitive interviewing, the psychologists call it. So the project has become more qualitative. I will also use a delphi panel to help me with final edits. The question is, what comes first, the delphi or the think-aloud interviews…. Hmmm.
- I’m becoming more interdisciplinary in my thinking. Strange since I’ve been immersed in the nursing world for all my courses and my teaching but what I am doing is not just for nursing. I’m discovering quickly that my work will spread further if I quit spinning it for nursing journals. I published the questionnaire paper and it was really good. The theory paper is awesome and I called it a theory for nursing education but…… it is a theory for all disciplines. It’s almost too bad that I sent it to a nursing journal but I also had some bones to pick with nursing and their writing publications so it is OK there. I’ve published a few other studies that have had some interesting findings and I’ve had more than one moment of being ready to lose my shit with some of the overly structured rules attached to some nursing journals. I nearly pulled one submission recently because of that. I had a great journal choice in educational psychology all picked out as my target for resubmitting and then when I went to read the paper for fit, it was all nursing this, nursing that.…… and it was going to be more work to remove the nursing spin than I was willing to do. I just want the damn paper published. I fear that the psychology people doing work in writing self-efficacy won’t find my papers in order to cite them. They will be unlikely to search CINAHL for this topic — for good reason.
- My study needs to be about more than about undergraduate writing. I was going to only interview undergrads but the fact of the matter is that I do want the questionnaire to be applicable to research on grad students as well. I also don’t want the questionnaire to be only applicable to nursing education. It needs to be interdisciplinary.
I need to be thinking about writing my research proposal soon even though I am about a year away from being ready to move to that stage. I’ve written now 4-5 papers that have required me to summarize and present a review of the literature on writing self-efficacy. It is going to be tough to find yet another way to write about the same findings without self-plagairising.
I still have a lot of reading left to do. The pile in the photo is all the articles that I have collected since summer of things I want to read. Some of those articles are about construct validity in writing and assessing writing outcomes so I hope to fit them into my work this term. Hence, since I often focus this blog, and my Twitter on what I am currently focusing on, there may be a little bit of a flavour change in what I write about for the next three months as I explore measurement, and hopefully, measurement as it relates to writing.
*In some ways, I would love to change my topic. I have been introduced to all kinds of shiny things that have grabbed my passions — eg. Narrative Inquiry, for one. But I have a committee now set up to get me through a measurement project so I carry on. And, this IS the next step in my work, this tool development. The big qualitative study will come after.