When the University of Michigan School of Public Health (MPH) implemented faculty activity reporting software to replace its internal faculty salary merit review system, it also took on a far more ambitious project: powering analytics with faculty activity data. The result is visualizations that represent both the quality and quantity of faculty’s teaching, research and service. Sam Russell, Business Analyst with Michigan Public Health, shared the school’s journey from internal system to visualizations of faculty performance at our recent User Group. In the last post, we explored MPH’s rapid Activity Insight implementation. Here, you’ll learn how MPH uses faculty activity data from Activity Insight in Tableau dashboards, showing context, ranking and analytics.
Turning Data Into Visualizations
From the beginning of its Activity Insight implementation, MPH’s vision extended beyond data and reporting. The final merit report for each faculty member is produced in Tableau, where the data can also be placed in context that’s visual and intuitive, with drill-down capabilities for closer examination of data when needed.
Russell detailed an example: a publication record. “We wanted a way to define and measure the value of a publication record,” Russell said. “The position of authorship matters; so does the journal itself.” MPH started with author position, which it captures in Activity Insight, and used calculations based on the Thomson Reuters Journal Impact Factor and other factors to arrive at a publication score. “The publication score rolls up into a research score, which rolls up, along with teaching and service scores, into the overall faculty merit score,” Russell said. The rolled-up data is then used as a primary data source to generate analytics.
Finding the Right Metrics
The Thomson Reuters Journal Impact Factor gave MPH’s publications score a well-researched foundation. Fortunately, with visualizations, MPH can use heat maps to check whether the metrics within its process for teaching and service create a score that makes sense. “We took the dean’s logic on how to weight faculty activities and put it into an automated, consistent model,” Russell said.
For example, MPH recognized that student course evaluation responses were influenced by class size, so that was built into the model to weight course evaluations. “If you find someone outside the norm, that’s where you start conversations,” Russell said. “If an individual is below expectations, you can have a corrective conversation. If someone is successfully achieving high ratings in large classes, you can model their teaching behaviors and share them.”
Surfacing Actionable Information
MPH went beyond analysis of individual faculty. “We developed a heat map of all individuals sorted by publication contribution score so we could rank them by percentile relative to others in their department or the school,” Russell said. This allows deans and faculty themselves to see how an individual compares to peers in the institution. “We spent time discovering our data—what we already have in Activity Insight. We discovered that we could identify high performers. These are people you want to generate discussions with to pick up on what they’ve been doing and what they think is successful,” Russell said.
Faculty with the same score may get that number in different ways. For example, they might get the same score for four or five articles in higher-impact journals as someone else got for 25 publications in lesser journals. “But if you can get the same score, why do 25? Talk with those who are more efficient and find out how they do it,” Russell said.
Quality Data Is the Ballgame
Russell credits Activity Insight with the increasing data quality and quantity that allows MPH to create insight through analytics. “We’ve moved away from manual processes to do more value-added and strategic mission work,” Russell said. “Now we can provide the relative value. A number is just a number. The value comes from context.” Using analytics, MPH creates rankings, projections, forecasts and exceptions they can use to better understand how individual faculty as well as the institution are performing.
“When we look at a number, we ask, ‘So what?’ and ‘What more?’” Russell said. “Reports are great, but what more can you do to aid decision-making processes? What does your faculty activity data mean in the process you’re aiding?”
Value to Many Stakeholders
The analytics MPH created in Tableau have value across the institution, Russell noted. Because the data is all there, administration can drill down to better understand the numbers on an individual or department.
And MPH plans to share some of the data with faculty. “Eventually, faculty will have access to information that helps them better understand where they’ve been, where they are compared to peers from the department and peers of the same rank across the institution,” Russell said. Faculty won’t be able to look at others’ individual data, but will be able to see overall data and where they fall in the spectrum. “It’s designed to give them insight,” Russell said.
MPH’s visualizations combine all the data you can find in an annual report and takes it farther, allowing them to WHAT data, surface new insights, bringing interesting and actionable things to light. “The key to data visualizations is that they aren’t just pretty pictures,” noted Andy Glassman, Digital Measures Back End Architect. “The drilldowns show how you got a number or score. As the faculty being reviewed, you’re not just a number, so it’s important to be able to show how you got there.”
MPH implemented Activity Insight rapidly, but plans to expand its use, from additional screens for capturing activity data to using it for additional processes. It also plans to calibrate its service score. “Service is the most challenging in terms of evaluation on an impact level,” Russell said. “For example, some committees in the school carry more weight than others.” By creating heat maps, scatter plots and other visualizations, MPH is able to assess whether its current metrics accurately reflect faculty contributions. Eventually, its analytics will also be used as a forecasting tool.
Are you interested in using Activity Insight data in Tableau or other visualization tools? We’d be happy to help, so contact us here.