Backtracking Alumni: Using Institutional Research and Reflective Inquiry to Improve Organizational Learning

In a recent blog post, we shared practical tips for developing an alumni tracking program to assess students’ employment outcomes. Alumni tracking is an effective tool for assessing the quality of educational programs and helping determine whether programs have the intended impact.  

In this post, we share the Backtracking technique, an advanced approach that supplements alumni tracking data with students’ institutionally archived records. Backtracking assumes that institutions and programs already gather student outcomes information (e.g., employment, salary, and advanced educational data) from alumni on a periodic basis (e.g., annually or every three years).  

The technique uses institutional research (IR) archives to match students’ employment outcomes to academic and demographic variables (e.g., academic GPA, courses taken, grades, major, additional certifications, internships, gender, race/ethnicity). By pairing student outcomes data with academic and demographic variables, we can contextualize student pathways and explore the whole pathway, not just a moment in time. 

Figure 1 shows an example of the Backtracking technique for two-year Associate of Arts (AA) and Associate of Science (AS).  

Figure 1. Backtracking Technique for AA/AS Programs 

Figure 1 illustrates three data collection layers. Layer 1, Institutional Research College Data, provides student completion data, academic history, and contact information. Advanced– and transfer-degree data are also available through the National Student Clearinghouse, which can reveal the major that a former student (or graduate) entered after completing the AA/AS degree. Layer 2, Alumni Transfer & Employment Data, includes student employment and advanced–degree information self-reported in alumni surveys.  

Layer 3, Pathway Explanatory Data, embeds a qualitative component within the Backtracking technique in order to let alumni explain their undergraduate experiences. This layer helps us understand what happened during and after college. Most importantly, it lets us identify the critical junctures that students faced and the facilitators and hindrances that allowed students to overcome (or that caused) setbacks during these difficult periods.  

To provide alumni with the best opportunities to share their experiences, we use IR archives to formulate questions based on key facts about students’ experiences. For example, if IR records show that a student transferred from college A to university B, we may ask the student about that specific experience. For a student who failed Calculus 1 once but passed it on the second try, we may ask what allowed that success. 

Although individual student pathways are useful, we can also stratify these data by race and gender (or other factors) and then aggregate them to better understand student groups. We demonstrate how we aggregate the pathways in this short video. 

The Backtracking technique requires skilled personnel with technical knowledge in IR and data collection and analysis or an Academic IR (who possesses both IR and research skills). Investing in such skill and knowledge is worthwhile:   

• • Institutional research is powerful when used for formative and internal improvement and for generation of new knowledge.  
  • Findings about former students using the Backtracking technique can provide useful information to improve program and institutional services (e.g., advising, formal practices, informal learning opportunities, etc.).  
  • Looking back at what worked or failed for past students can inform current practices and serve as a source of institutional learning.  

References: 

Jones, F. R., Mardis, M. A. (2019, May 15). Alumni Tracking: The ultimate source for evaluating completer outcomes [Blog post]. Retrieved from https://www.evalu-ate.org/blog/jones2-may19/