Measurement, Analysis, and Knowledge Management (Category 4)
Build STC's scorecard of leading and lagging measures with comparisons, run a review cadence that leads to action, project future performance, and manage data quality, security, and knowledge, from shared measure definitions to master course shells and governed AI tools.
The Foundation of the System
Category 4 (90 points) sits under everything else in the framework diagram. Without reliable measures, analysis, and knowledge, management by fact isn’t possible.
- 4.1 Measurement, Analysis, Review, and Improvement of Organizational Performance (45): which measures you track, where your comparative data comes from, how you keep measures agile, how leaders review performance, how you project future performance, and how reviews lead to improvement.
- 4.2 Information and Knowledge Management (45): how you manage knowledge and share best practices, ensure data quality, availability, and security (including cybersecurity), and pursue innovation.
From Data to Decisions
Choosing Measures
STC tracks about 20 key measures. A few from its scorecard:
| Measure | Type | Frequency | Comparison | Linked to |
|---|---|---|---|---|
| Week-2 LMS login for new students | Leading | Weekly | Own trend | SO2 |
| Gateway course DFW rate | Leading | Quarterly | Peer institutions via a shared consortium | SO2 |
| First-year persistence, online | Lagging | Quarterly / annual | National Student Clearinghouse | SO2 |
| Six-year completion, part-time transfer students | Lagging | Annual | IPEDS Outcome Measures | 7.1 |
| Graduate employment in field within six months | Lagging | Annual | Labor-market data; peer programs | 7.1, SO1 |
| Adjunct onboarding completion | Leading | Quarterly | Own target | SO4 |
| Adjunct engagement index | Lagging | Annual | Pulse benchmark from survey vendor | SO4 |
| Cost per credit hour | Lagging | Quarterly | Other schools in the university; instructional-cost benchmarking studies | SO5 |
Leading measures give early warning that you can act on this quarter. Lagging measures confirm whether outcomes improved. A scorecard with only lagging measures tells you what went wrong a year too late.
Timing is what separates them. For a new student, the measures arrive in this order:
Measurement agility means you can add or change measures quickly. When STC launched AI tutoring, it added “tutor sessions per student” and “help-desk tickets on AI tools” in the same quarter.
Reviews That Lead to Action
4.1b asks how senior leaders review performance and capabilities, and 4.1c asks how reviews lead to improvement and how you project future performance.
At STC’s monthly leadership review, each action-plan owner reports status against projection in one slide: on track, at risk, or off track. Only at-risk and off-track items get discussion time. Each review ends with a list of decisions: what changes, who owns it, and when it’ll be checked.
Projections connect measurement to strategy. In the quarterly review, the team compares actual persistence with the projection in the SO2 action plan. If the gap is growing, it adjusts the plan (2.2b) or reallocates resources.
Managing Knowledge
Knowledge at a school is easily lost, because so much lives in individual instructors’ heads and course sections. STC manages it deliberately:
- Master course shells. Every section is built from a shared master. Improvements made in one section are reviewed and added to the master, so the next 15 sections benefit.
- Teaching circles. Full-time and adjunct faculty meet each quarter by course cluster to share what worked.
- Retirement handoffs. Retiring faculty co-teach with their successor for a quarter and record walkthroughs of key labs.
- Best-practice transfer. When one program’s early-alert follow-up rate hit 95%, its process was documented and adopted by the other programs.
Data Quality, Security, and AI Tools
- One definition per measure. “Persistence” means the same thing in the dean’s dashboard, the program review, and the NWCCU report. The definitions live in a shared data dictionary.
- Access by role, following FERPA: instructors see their own sections, advisors see their caseload, and leaders see aggregates.
- Cybersecurity of student and research data is managed with the university’s IT security office and tested through regular phishing exercises and access reviews.
- AI tools are reviewed before adoption: what data they use, where it’s stored, whether vendors can train on it, and whether outputs are checked for bias across student segments.
Common mistake: STC builds an impressive dashboard with 70 charts. Leaders look at it in meetings and say “interesting,” and nothing changes.
Measures earn their place by driving decisions. For each measure, ask: Who reviews it, how often, against what target and comparison, and what do they do when it’s off? If no one can answer, drop the measure or fix the review.
Key Takeaway
Category 4 is the foundation. Choose a small set of leading and lagging measures linked to objectives, each with a comparison. Review them on a defined cadence that ends in decisions, and project future performance. Manage knowledge through master shells, teaching circles, and handoffs. Keep data accurate, consistently defined, secure, and governed, including any AI tools that use student data.
For your school, list ten key measures. For each, fill in:
| Measure | Leading or lagging | Frequency | Comparison source | Who reviews it | Linked objective |
|---|
Check the result:
- Do you have at least three leading measures?
- Does every measure have a comparison other than your own past performance?
- Does every strategic objective have at least one measure?
1. Which is a leading measure for first-year persistence?
Early LMS engagement gives a warning you can act on in the current quarter. The others are lagging.
2. What does 'measurement agility' mean?
Measurement agility is the ability to respond to rapid or unexpected change with new or adjusted measures, as STC did for AI tutoring.
3. Why do master course shells support knowledge management?
A shared master captures improvements and best practices so knowledge isn't trapped in individual sections.
4. STC has a 70-chart dashboard that leads to no changes. What is the core fix?
Measures matter only when someone reviews them against targets and acts on what they show.