Interview with Kimberlee Everson, Ph.D. - Chair of the Carnegie Project on the Education Doctorate’s Dissertation in Practice Award Committee and Associate Professor at Western Kentucky University
About Kimberlee Everson, Ph.D.: Kimberlee Everson is an Associate Professor of Research Methods for The School of Leadership and Professional Studies at Western Kentucky University (WKU). She is a specialist in statistics, psychometrics, and other quantitative methodologies that seek to identify and address barriers to strong student learning outcomes. As Associate Professor, she mentors students of WKU’s EdD in Educational Leadership with regards to selecting and implementing appropriate research methodologies to investigate their chosen problem of practice.
In addition to her research and teaching roles at WKU, Dr. Everson serves as Chair of the Carnegie Project on the Education Doctorate’s (CPED) Dissertation in Practice Award Committee. As Chair, she confers with her fellow Committee members to determine which submitted dissertations receive this prestigious award. Dr. Everson is an advocate for context-embedded EdD dissertation topics that focus on direct action within students’ workplace, and which also incorporate improvement science as well as other pragmatic methodologies. She has written extensively on the topic of the “ideal” EdD dissertation, and has a book on this subject forthcoming in 2024. In addition, Dr. Everson has also begun to investigate the impact that Artificial Intelligence (AI), including Large Language Models (LLM), are having on EdD students’ learning and their work on their dissertation.
Dr. Everson received her PhD in Educational Inquiry, Measurement, and Evaluation; her Master of Science in Statistics; and her Bachelor of Science in Statistics from Brigham Young University.
Interview Questions
[OnlineEdDPrograms.com] You are currently an Associate Professor of Research Methods for The School of Leadership and Professional Studies at Western Kentucky University (WKU). Could you elaborate on your areas of research expertise, as well as the courses you teach as Associate Professor?
[Dr. Kimberlee Everson] I am a statistician and research methodologist, and so my teaching and research focus on research methods, particularly quantitative methods. I teach general research methods, statistics, and intermediate and advanced quantitative methods courses. In addition to quantitative methods, I’m a fan of mixed methods, and I very frequently use a methodology called Q sorting, which has elements of both quantitative and qualitative methods.
My first research interest was in value-added modeling, which is a set of methods used to attempt to link student learning gains to teachers and/or schools. I began that interest with a very skeptical lens, and I’ve maintained that skepticism. My two most significant publications on that topic are a literature review in Review of Educational Research, focusing mostly on statistical issues, and a paper in Harvard Educational Review. The Harvard Educational Review paper argued that any statistical model used to link student outcomes or growth to teachers should use the statistical effect called “the treatment effect on the treated.” This approach looks at how well a teacher’s students performed given the actual traits of that group rather than compared to all students. It holds teachers accountable for teaching well the types of students actually assigned to them.
I still hold a skeptical point of view on value added modeling and similar models. I do not believe they can adequately capture context, the various forces that contribute to student learning in any classroom, and teacher success in less measurable ways. At the moment I am more interested in teacher beliefs about evaluation and accountability processes. My research encompasses beliefs and belief change processes in other areas as well, such as surrounding artificial intelligence and even religious change.
In general, I believe that much of the good (and bad) that happens in classrooms is unmeasurable. I’m observing that many of the efforts to require teachers to document what they are doing, to evaluate their teaching, and/or to measure their success do more harm than good. It takes away time from preparation, teaching, and leading teachers in ways that might actually have impact on student learning—and the interpretations from most of those efforts have serious validity issues.
There isn’t a magic solution to measuring the impact of educational interventions, teachers, or schools. Even if we could validly link those factors to student test scores, there is so much more to learning, and child growth and development, than test scores.
As a parent I have seen horrible teachers, based on the impact they had on a child emotionally, who did probably impact test scores better than some other more loving teachers. But how do you measure emotional impact? How do you measure teaching a child to love learning? How do you measure being so effective that a child wants to be like you when they grow up?
I believe administrators generally have an accurate idea about who the wonderful teachers are and who the problem ones are, but I don’t know that we will ever find a way to measure that objectively and fairly. And I believe that by trying to do so we are burning out all the teachers, even the wonderful ones.
[OnlineEdDPrograms.com] In addition to your role as Associate Professor, you are Chair of the Carnegie Project on the Education Doctorate’s (CPED) Dissertation in Practice (DiP) Award Committee. What are your responsibilities as Chair? What inspired you to join CPED, and how have your collaborations with CPED impacted your own practice and WKU’s EdD program?
[Dr. Kimberlee Everson] I became involved with CPED at the encouragement of my university. Because I do a lot of dissertation committee work and teach students about the dissertation in my courses, I was interested in joining the DiP awards committee and volunteered. The award was created to locate models of ideal dissertations, from CPED’s perspective, and to motivate programs and students to follow those models.
Over the years there has been a lot of work done by the committee to revise the rubric for the award so that it best reflects what we hope to see. For the last two years I served as a co-chair of the committee and have focused on better analysis of the dissertation ratings done by the committee members to increase inter-rater reliability and assure ourselves that we were choosing the best dissertation as the winner.
The three previous chairs and I have co-edited a volume that is now in press with Myers Education Press that is a guide to the DiP based on our work on the committee and it thoroughly discusses what we consider to be a good dissertation. In short, we are looking for a dissertation that is deeply embedded in the student’s own work context, that uses theory from the academic literature effectively, that designs and implements a change or intervention of some type, and that carefully evaluates the results of that intervention and change. We want to see models of making theory-based change that does real good by a real leader within a real applied setting. In addition, we are looking for dissertations that are focused on issues surrounding equity. We are looking for true practitioner-scholars who can lead educational institutions for change.
Being involved in this effort has focused my efforts as a research methods professor toward creating higher quality dissertations, particularly improvement science dissertations. I have advocated for using the CPED model in our EdD program and helped design dissertation templates and guides as well as research methods courses that align.
[OnlineEdDPrograms.com] CPED first presented its Dissertation in Practice Award in 2012. From 2012 to now (2023), how have you seen the EdD dissertation evolve, in terms of purpose, format, content, and structure? How do you envision the dissertation evolving further in the next decade?
[Dr. Kimberlee Everson] I am particularly interested in dissertation formats and quality within member programs. I feel that, while we have seen more dissertations submitted to the award that reflect CPED dissertation in practice principles, we still see a lot of traditional dissertations.
I have also been involved in the Improvement Science CPED Improvement Group (CIG) and pay close attention to comments made in various convening sessions. I think a lot of institutions are not really sure what the dissertation in practice is or do not know how to get started with it in their programs. There is still a lot to do.
In the next decade we will have particular challenges in these efforts as we also deal with generative AI. There were four AI-and-the-dissertation related sessions at the CPEDs in-person convening in October 2023, and I presented a 5th session on the topic at the virtual convening that followed.
The “ideal dissertation” is going to shift in form as we learn what are the optimal uses of AI within the dissertation process. I have written a chapter on the topic for CPED’s book series, in a volume called “The Importance of the Dissertation in Practice” that should be available spring 2024. In addition, I am working towards publishing a complete book on the topic as soon as possible, hopefully in 2024-25.
[OnlineEdDPrograms.com] In speaking with CPED’s Executive Director Dr. Jill Perry, we learned how CPED upholds and encourages impact-focused scholarship, and we discussed different applied research approaches, including action research, improvement science, and evaluation. Could you elaborate on these and other applied research approaches that are particularly useful for DiPs, and why CPED advocates for these methodologies when addressing a Problem of Practice (PoP)?
[Dr. Kimberlee Everson] The approaches used in DiPs need to be appropriate for context-embedded studies, particularly those that are intended to improve a problem of practice. Because they are embedded in context, descriptive statistics are generally more important than inferential statistics, and qualitative or mixed methods approaches that are focused on practical improvements within specific contexts are more appropriate than those intended to develop theory.
I feel like pragmatic paradigms that combine the most appropriate method for each stage of the study are appropriate. A student may do a descriptive study using a survey in order to understand the root causes of their problem of practice, or they may use a qualitative case study approach. When designing an intervention or change, they may use literature and theories of action. Quantitative metrics and dashboards might be useful to track progress as the intervention is tested, or they may use a general qualitative approach with interviews or focus groups.
While we want the student to use theoretical or conceptual frameworks, “what works” is the primary concern. Potentially any research method might be useful in understanding the problem of practice and in evaluating any changes or interventions the student tests, but we generally encourage students to use simpler methods when possible.
[OnlineEdDPrograms.com] OnlineEdDPrograms.com is developing an unprecedented dissertation search tool and database that will enable educators and EdD students to find cutting-edge scholarship their peers are conducting to improve education outcomes in diverse contexts. In our research for this database, we found that many dissertations do not specifically call out the research approach they are using. When it comes to successfully completing a dissertation, at what point do students consider and choose their primary research approach?
[Dr. Kimberlee Everson] I would say that the best dissertations in practice that are submitted for the award almost universally decide on the PoP first, and it is a PoP they are dealing with in their individual practices or careers. Sometimes the PoP is triggered by a specific event or interaction they have had. They are really concerned about a problem they see, and then that problem drives the methods.
I do think that specific instructors and/or universities emphasize specific approaches. For example, my university embraces improvement science. I have seen others that use a lot of action research. But when the program does not push a specific approach, the approach the student chooses often is triggered by one of the following: a study the student read that they wish to model theirs after, the advice of their committee chair or methodologist, or the emphasis of coursework.
In my program, for example, we only recently have shifted entirely to improvement science. When I serve as a methodologist or chair, I have a conversation with the student and try to understand what they really want to find out or achieve. That conversation generally leads to a methodology, but my recommendations are biased as I have favorite paradigms and methods. Of course, having a broad knowledge of research methods helps students but primarily as consumers of research. Few students understand research methods well enough that they can, on their own, choose an appropriate methodology.
[OnlineEdDPrograms.com] Where do you see the EdD going in the next 20 years–how will it be different from its current iteration, what curricular components will be present that are not present now, and how will it combine professional knowledge, technical skill, and evidence-based approaches to improving education outcomes? What challenges do you see the EdD encountering as it seeks to change and improve?
[Dr. Kimberlee Everson] The biggest changes over the next 20 years will be related to AI. Students will use AI in ways that we cannot even imagine now to learn, to examine and solve problems of practice, to be involved in advocacy, and to contribute to their fields.
I believe that dissertations will evolve to become “projects” that may be presented in a variety of ways other than the traditional written dissertation, and I think we will more completely understand that what matters is what a student understands, does, and can communicate in order to improve the world. I really hope that equity issues improve dramatically and that all students have equal access to tools, education, and leadership opportunities.
CPED’s vision is very appropriate, and WKU fully embraces that vision. I think CPED (and WKU) will see some shifting toward even more inclusion, more embracing diversity, and more embracing technological tools. I hope that we will move to a place of, not just studying problems and teaching students to lead within local contexts, but to becoming advocates for change in the larger societies.
Yes, change needs to begin at the local level, but we are learning so much that we can share beyond that circle. It would be wonderful if every school district is led by CPED-trained scholar-practitioners and leaders, but I would also love to see us create national and international leaders and change agents. Let’s not just change the EdD; let’s change the world.
Thank you, Dr. Kimberlee Everson, for your excellent insight into EdD dissertation methodologies, the importance of context-specific problems of practice, and the burgeoning impact of AI on how both education leaders and students learn!
