Better Surveys and Reports Yield Results Future Teachers Can Use

The Challenge: Turn Data-Heavy Policy Into Valid and Reliable Consumer-Friendly Reports

In 2010, the Texas State Legislature passed a bill requiring the development of a data-rich accountability system for educator preparation programs (EPPs). As part of this system, the Texas Education Agency (TEA) was required to develop and administer an online survey to principals of new teachers. Initial obstacles to survey implementation included:

• A need for well-defined data entry rules to improve data reliability
• Technical problems reported by end users responding to the online survey
• Aggregate reporting that showed little variance between EPPs

In response, TEA partnered with the Texas Comprehensive Center (TXCC) for support in implementing the principal survey to rate the quality of new teachers’ preparation.

The Strategy: TXCC Staff Modeled High-Leverage, Capacity-Building Solutions to Improve Data Quality and Utility

TXCC consulted with TEA to establish project management best practices across departments

• Moved from ad-hoc to regular meetings with all key staff “at the table” to address data quality and technical problems.
• Defined clear roles and responsibilities for the project team, establishing ownership of critical tasks.
• Mapped the survey process from beginning to end, ensuring that individuals understand their role.

TXCC provided training and demonstrations of data analysis and visualization techniques

• Revised timelines to clarify the time required to conduct large-scale data analysis.
• Illustrated how data can be disaggregated to reveal meaningful differences in EPP ratings.
• Applied data visualization techniques to enhance TEA’s ability to communicate clearly with external audiences.

Impact: Better Processes and Better Reports

The Accountability System for Educator Preparation (ASEP) team moved from working reactively and in isolation to collaborating across departments, yielding clear business rules to produce more reliable data. Project leadership identified limitations to the survey process and allocated resources strategically.

Staff increased their understanding of effective data analysis and visualization (see reverse) that can inform policy and, more importantly, provide consumers and future teachers needed information to differentiate between EPPs.

System Change: Planning Gets More Rigorous, Integrated

Strategies used in the ASEP project, new project management and data analysis skills, bolstered the development of the Texas State Teacher Equity Plan. Using new strategies and tools, TEA staff streamlined efforts, made sure adequate resources for data analysis were in place, examined gaps and subpopulations and ensured data were reported to stakeholders using innovative visualization techniques.