A Mixed Methods Researcher, Evaluator, and Strategist.
Organizational Strategy
Case Study
Project Overview
XSEDE is a geographically distributed project with nearly 20 sites that work together to provide free advanced computing resources to its users. In its second phase of funding, the organization wanted to know how to better measure and track progress towards their goals and objectives.
Role:
Co-led the review and revision of existing key performance indicators and metrics, oversaw design of new reporting system, aligned quantitative and qualitative measures with project goals and objectives, engaged cross-functional leads to ensure metrics were meaningful, impactful, and useful for decision-making, identified data sources, and reconstructed historical data.
Methodology:
Adopted a participatory approach to engage cross-functional leadership in the review and revision of project-wide metrics. Work was conducted in four key phases detailed below.
Phase 1:
Comprehensive review and rating of all metrics for relevance, validity, stability, importance, and decision-making utility. Incorporated feedback from previously conducted surveys regarding the original performance measurement system. In-person retreat with strategy and evaluation experts to compare and discuss assessments. Aggregation and compilation of initial review recommendations.
Phase 2:
Project area reviews with each cross-functional lead to discuss recommendations from Phase 1, record feedback, and modify proposed set of metrics.
Phase 3:
Final review, presentation, and approval by project leadership followed by funding agency approval.
Phase 4:
Internal process development and implementation for ongoing data collection and reporting on qualitative and quantitative metrics with real time access for program managers via Tableau.
Results:
This process resulted in a streamlined system that reduced reporting metrics by 57% and boosted data-driven decision-making throughout the project by 30%.
Outcomes:
By aligning manager priorities with organizational goals, we were able to develop meaningful metrics that demonstrated impact to the funding agency while informing routine decision-making. This work has been presented in multiple invited talks at the National Science Foundation, conferences, and academic seminars.
