Using a Systems Orientation in Your Evaluation

In my earlier blog about CLIPs (Communities of Learning, Inquiry, and Practice), I mentioned that we adapted the CLIPs process to focus on building the capacity of evaluators to use a systems orientation in their evaluation. We called this one ECLIPS—Evaluation Communities of Learning, Inquiry, and Practice about Systems.¹ The ECLIPS process built the capacity of evaluators of STEM education programs (mainly ITEST and ATE) to use a systems orientation in their evaluations. You can learn more about the ECLIPS process in Evaluation Communities of Learning, Inquiry, and Practice about Systems (ECLIPS) Approach and Webinar Example, which is posted on our InSites website.

In ECLIPS, STEM education evaluators learned and applied three systems concepts to evaluations they were engaged in. The concepts were (a) understanding a system as a configuration of interacting, interdependent parts that achieves something; (b) recognizing and distinguishing among organized, unorganized, and self-organizing system dynamics; and (c) identifying patterns within systems based on attending to a system’s boundaries, relationships, and perspectives. We found that evaluators could incorporate systems thinking into their evaluation, even if the evaluation was already underway.

Here are three articles that will give you more information about how to apply a systems orientation to evaluation. Keep an eye on our website because we are planning to introduce more ideas about a systems orientation to evaluation in the future.

Using Complexity Science Concepts When Designing System Interventions and Evaluations
This article provides an overview of basic concepts related to complex adaptive systems (CAS). This is a very useful concept. It has helped many of us evaluators better understand what to try to control and what to simply follow to understand a project’s natural evolving patterns.

ZIPPER: A Mnemonic for Systems-Based Evaluation
ZIPPER is a mnemonic that gives you the basic idea of what to think about when you are doing a systems-oriented evaluation. It starts with “Zooming in and out.” Check out the article to see the other five actions to enrich your evaluation through attention to the systems that provide the context for your ATE project.

Habits of a Systems Thinker
This graphic overview reminds us of how to think more systemically about our work.
These articles are available on the InSites website, along with examples of how ECLIPS members used a systems orientation in their evaluation work. Take a look at some examples from Karen Peterman, David Reider, and Ginger Fitzhugh.

I love talking about using a systems orientation in evaluation. Feel free to contact me if you’d like to discuss these ideas further.

¹ This work was funded as an exploratory study through a Promoting Research and Innovation in Methodologies for Evaluation (PRIME) grant from the National Science Foundation (#1118819).