Our Take on AI’s Role for Professional Evaluators

Many of us evaluators are old enough to have seen firsthand the monumental shifts brought about by personal computers in the ’90s and later by smartphones. These innovations drastically changed how we approached work. Our article, “Artificial Intelligence (AI) for Evaluators: Opportunities and Risks,” published in the latest issue of Journal of MultiDisciplinary Evaluation, addresses the technological advances in AI in the context of the evaluation discipline. Here we give a brief overview of that article, summarizing what we see as potential implications of AI for our fellow professional evaluators.

Reflecting on personal experiences, I (Aaron) recall a moment from the late 1990s when my father introduced a Windows 95 computer to my grandfather’s law firm. While my grandfather was skeptical, deeming it a mere fad, my father saw its potential. He harnessed its capabilities to streamline processes, reducing the need for administrative staff. This memory serves as an analogy for AI’s looming impact¾a transformative force ready to redefine our professional routines.

The primary focus of our article is exploration of the possibilities AI holds for professional evaluators, addressing questions such as: How can we integrate it into our workflows? How might it alter our roles? It’s crucial for evaluators to recognize that AI is more than just a tool; it’s a game-changer. By embracing AI, we believe evaluators can heighten the precision, speed, and depth of their work. AI has unparalleled potential to analyze massive data sets, spot intricate patterns, and offer nuanced insights.

AI is clearly not without challenges and risks, such as job loss, concerns about data security, and the moral quandaries surrounding AI’s decision-making. We stress the importance of maintaining a critical stance. While we should harness AI’s capabilities, it’s vital not to become complacent or place undue trust in its outputs. Algorithms, no matter how advanced, have biases and inaccuracies that need vigilant oversight.

Interestingly, the creation of this blog post showcases the practical applications of AI. The content was crafted with the assistance of ChatGPT, an advanced AI model. By letting it read our article and prompting it to summarize it in the form of a blog post, we were able to collaboratively generate this comprehensive piece. It exemplifies the synergy between human thought and AI capabilities, underscoring our belief in AI as a valuable tool when used responsibly and creatively.

For those evaluators eager to explore AI, here are three essential tips:

1. Start Small: Begin with user-friendly tools and applications tailored for beginners to get a foundational grasp of this technology. By starting with simple use cases like letting an AI assist you in drafting an email, you can get comfortable with the benefits and limitations without attempting higher-risk applications.
2. Use Caution: AI can often hallucinate, generating falsehoods that seem credible. If you’re tempted to let ChatGPT create a literature review for your next proposal, beware the rampant fake citation.
3. Ethical Considerations: Always be conscious of the ethical dimensions when employing AI, ensuring transparency and respect for privacy. Always ensure that you, your clients, and your employer are on the same page about what constitutes ethical use, especially when it comes to data privacy.