Many of us evaluators are old enough to have seen firsthand the monumental shifts brought about by personal computers in the ’90s and later by smartphones. These innovations drastically changed how we approached work. Our article, “Artificial Intelligence (AI) for Evaluators: Opportunities and Risks,” published in the latest issue of Journal of MultiDisciplinary Evaluation, addresses the technological advances in AI in the context of the evaluation discipline. Here we give a brief overview of that article, summarizing what we see as potential implications of AI for our fellow professional evaluators.

Reflecting on personal experiences, I (Aaron) recall a moment from the late 1990s when my father introduced a Windows 95 computer to my grandfather’s law firm. While my grandfather was skeptical, deeming it a mere fad, my father saw its potential. He harnessed its capabilities to streamline processes, reducing the need for administrative staff. This memory serves as an analogy for AI’s looming impact¾a transformative force ready to redefine our professional routines.

The primary focus of our article is exploration of the possibilities AI holds for professional evaluators, addressing questions such as: How can we integrate it into our workflows? How might it alter our roles? It’s crucial for evaluators to recognize that AI is more than just a tool; it’s a game-changer. By embracing AI, we believe evaluators can heighten the precision, speed, and depth of their work. AI has unparalleled potential to analyze massive data sets, spot intricate patterns, and offer nuanced insights.

AI is clearly not without challenges and risks, such as job loss, concerns about data security, and the moral quandaries surrounding AI’s decision-making. We stress the importance of maintaining a critical stance. While we should harness AI’s capabilities, it’s vital not to become complacent or place undue trust in its outputs. Algorithms, no matter how advanced, have biases and inaccuracies that need vigilant oversight.

Interestingly, the creation of this blog post showcases the practical applications of AI. The content was crafted with the assistance of ChatGPT, an advanced AI model. By letting it read our article and prompting it to summarize it in the form of a blog post, we were able to collaboratively generate this comprehensive piece. It exemplifies the synergy between human thought and AI capabilities, underscoring our belief in AI as a valuable tool when used responsibly and creatively.

For those evaluators eager to explore AI, here are three essential tips:

  1. Start Small: Begin with user-friendly tools and applications tailored for beginners to get a foundational grasp of this technology. By starting with simple use cases like letting an AI assist you in drafting an email, you can get comfortable with the benefits and limitations without attempting higher-risk applications.
  2. Use Caution: AI can often hallucinate, generating falsehoods that seem credible. If you’re tempted to let ChatGPT create a literature review for your next proposal, beware the rampant fake citation.
  3. Ethical Considerations: Always be conscious of the ethical dimensions when employing AI, ensuring transparency and respect for privacy. Always ensure that you, your clients, and your employer are on the same page about what constitutes ethical use, especially when it comes to data privacy.

About the Authors

Dr. Aaron Kates

Dr. Aaron Kates box with arrow

Lead Consultant, Effect X

Aaron holds a Ph.D. in Evaluation from Western Michigan University, a Master of Social Work, and a Bachelor of Science in Geography from Grand Valley State University. With eleven years in program evaluation, Aaron has worked for clients like the WK Kellogg Foundation and Calvin University. He is passionate about creating accessible evaluations that empower change-makers, using his vast knowledge and methodological diversity for tailormade solutions for unique programs and clients.

Kurt Wilson

Kurt Wilson box with arrow

EffectX

Kurt leads EffectX with extensive non-profit consulting and evaluation expertise, serving clients from local groups to large NGOs and foundations. Kurt holds a BS in Economics from The Colorado College, a Diploma in International Development from the University of London, and a Ph.D. in Interdisciplinary Evaluation from Western Michigan University. As an Affiliate Faculty Member at Regis University, he teaches in the Masters of Development Practice program.

Creative Commons

Except where noted, all content on this website is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Related Blog Posts

Nation Science Foundation Logo EvaluATE is supported by the National Science Foundation under grant number 1841783. Any opinions, findings, and conclusions or recommendations expressed on this site are those of the authors and do not necessarily reflect the views of the National Science Foundation.