Figures at Your Fingertips

In formative evaluation, programs or projects are typically assessed during their development or early implementation to provide information about how best to revise and modify for improvement. (www.austinisd.org)

 I’m Gordon Snyder and I’m currently the principal investigator of the National Center for Information and Communications Technologies (ICT Center). My experience as new ATE center PI back in July of 2001 did not get off to a very smooth start. With the retirement of the founding PI after three years, I was moving from a co-PI position and faced with making a number of decisions and changes in a short period of time.  We were right in the middle of the “dot com bust” and the information and communications technology field was in free fall. I knew our decisions needed to be data-driven, smart, focused, quick, and correct if we were going to continue to be a resource for faculty and students.

As a center co-PI during the boom times between 1998 and 2000, my role was focused on curriculum development and helping faculty learn and teach new technology in their classrooms and labs. I honestly did not understand nor pay much attention to the work our evaluator was doing – that was something the PI liked to handle, and I was perfectly fine with that.

In my new role as a PI, things changed. One of the first things I did was read the evaluation reports for the past two years. I found a lot of flowery complimentary language with little else in those reports – I recall using the term “pile of fluff” along with a few others that I won’t repeat here. I found nothing substantial that was going to help me making any decisions.

In August of 2001, I received our year 3 annual evaluation report and this one was even more “fluffy.” Lesson learned: Within a month I dismissed that evaluator, replacing that individual with someone more in tune with what we needed. Things were much better with the new evaluator, but I still found it difficult making intelligent data-based decisions.  I did not have the information I needed. There had to be a better way.

Fast forward to today: ATE PIs need even more access to valid, reliable, useful, evaluative data for decision making. This data needs to be available in real time, or close to real time throughout the funding cycle, more frequently than the typical annual evaluation reports. However, most PIs still simply do not have the time, resources, and expertise required to systematize the collection and use of this kind of information.

Logic models are one method that’s catching on to keep track of and use information to make formative data-driven decisions. I’ve been working on the FAS4ATE project (Formative Assessment for ATE) with Western Michigan University that will ultimately develop some logic model-based online tools to streamline data collection and more effectively scope and plan evaluation activities to include formative and summative processes. We’re early in the development process. Here’s a short video demonstrating our prototype of one of the tools.

Logic models are a great way to keep up with project work and more quickly and confidently make data-based decisions. If you’d like to learn more about this formative assessment project, contact me at gordonfsnyder@gmail.com