The Challenge: Turn Data-Heavy Policy Into Valid and Reliable Consumer-Friendly Reports
In 2010, the Texas State Legislature passed a bill requiring the development of a data-rich accountability system for educator preparation programs (EPPs). As part of this system, the Texas Education Agency (TEA) was required to develop and administer an online survey to principals of new teachers. Initial obstacles to survey implementation included:
- A need for well-defined data entry rules to improve data reliability
- Technical problems reported by end users responding to the online survey
- Aggregate reporting that showed little variance between EPPs
In response, TEA partnered with the Texas Comprehensive Center (TXCC) for support in implementing the principal survey to rate the quality of new teachers’ preparation.
The Strategy: TXCC Staff Modeled High-Leverage, Capacity-Building Solutions to Improve Data Quality and Utility
TXCC consulted with TEA to establish project management best practices across departments
- Moved from ad-hoc to regular meetings with all key staff “at the table” to address data quality and technical problems.
- Defined clear roles and responsibilities for the project team, establishing ownership of critical tasks.
- Mapped the survey process from beginning to end, ensuring that individuals understand their role.
TXCC provided training and demonstrations of data analysis and visualization techniques
- Revised timelines to clarify the time required to conduct large-scale data analysis.
- Illustrated how data can be disaggregated to reveal meaningful differences in EPP ratings.
- Applied data visualization techniques to enhance TEA’s ability to communicate clearly with external audiences.
Impact: Better Processes and Better Reports
The Accountability System for Educator Preparation (ASEP) team moved from working reactively and in isolation to collaborating across departments, yielding clear business rules to produce more reliable data. Project leadership identified limitations to the survey process and allocated resources strategically.
Staff increased their understanding of effective data analysis and visualization (see reverse) that can inform policy and, more importantly, provide consumers and future teachers needed information to differentiate between EPPs.
System Change: Planning Gets More Rigorous, Integrated
Strategies used in the ASEP project, new project management and data analysis skills, bolstered the development of the Texas State Teacher Equity Plan. Using new strategies and tools, TEA staff streamlined efforts, made sure adequate resources for data analysis were in place, examined gaps and subpopulations and ensured data were reported to stakeholders using innovative visualization techniques.
Old EPP Report
New EPP Report
The distributions of scores on each standard show the relationship between EPP X and EPPs across the state
Side-by-side bar graphs help readers compare EPP types
Displaying multiple indicators for each Standard 2 domain allows for differentiation