Using Data Analytics to Support High- and Low-Performing Schools

October 2018 Blueprint for Education.jpg

School districts are inundated with various types of school and classroom performance data – and not all types of data are suitable in understanding how to best provide differentiated supports to schools.

Data analytics such as growth data – or value-added (VA) data – can provide valuable insights into how well the instructional and leadership quality in classrooms or schools “grow” student learning year to year. Attainment data are scores typically used for providing a “snap shot” in time of how much a student has academically achieved as measured by standardized tests.

Combining these two data sources provides an insightful lens on how different types of schools are providing effective supports to teachers and students, and which schools may need additional or differentiated supports to improve or sustain their instructional efforts. In a study of 8 schools in Milwaukee Public Schools (MPS), we selectively sampled schools that varied across achievement and value-added scores (Table 1) to find out how different schools engage in data use practices.

Screen Shot 2018-10-23 at 1.58.53 PM.png

Two schools within each category were sampled and school-site interviews with teachers and principals were conducted to understand how they viewed their students, instructional quality, and leadership quality in schools (Table 2).

Screen Shot 2018-10-23 at 1.59.59 PM.png

High- and low-performing school personnel showed some key differences in their approach to data use for school and classroom performance management. For example:

  • High-performing schools articulated a “culture of data use and mindset of student growth” while low-performing schools focused on behavior rather than academic performance.

  • High-performing schools emphasized learning team collaboration and representation of various school leaders and teachers. These teams met on a regular basis to discuss and plan for specific goals.

  • Low VA/low attainment schools did not demonstrate this level of focus or team cohesion: Their meeting did not include an agenda, discussion of data or improvement planning, or plans for ongoing meetings and work.  In one learning team session in a high VA/low attainment school, the team struggled to interpret short-term reading assessment data and formulate a coherent strategy.

While not a generalizable study, these preliminary findings provide transferrable practices to use in districts and schools. Identifying the schools that are high performing using VA, then differentiating on their level of attainment provide insights on the types of supports needed for each school. Schools that score higher on VA grow students regardless of where students start academically. This is especially important for schools with students that score lower on attainment because it shifts the perception of how well the school is actually performing. Low attainment/low VA scoring schools can look to their demographic counterparts in other schools who are growing students in VA to find out what kinds of culture, mindsets, and data practices they are using to find gains.

District administrators can use these two types of data in tandem to understand patterns of performance across the schools in their districts. Using the lens of high/low growth (VA) versus high/low attainment (achievement scores) can help to stratify and further craft how school-based supports are delivered and utilized a cross a district (e.g., professional development practices on data use).

Data analytics is part of our Strategic Consulting and Technical Assistance Services. To find out more, contact sara@blueprintforeducation.org to discuss how to use data analytics in your strategic planning and school-based decision supports.

 

Full citation:

Kraemer, S., Geraghty, E., Lindsey, D., & Raven, C. (2010). School leadership view of human and organizational factors in performance management: A comparative analysis of high- and low-performing schools. In Human Factors and Ergonomics Society (Ed.), Proceedings of the Human Factors and Ergonomics Society Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society.

Applying Systems Thinking to Address Educational Inequities

Systems Thinking Educational Inequitiesjpg

Authors: Sara Kraemer, Blueprint for Education, and Don Gillian-Daniel, University of Wisconsin-Madison’s Delta Program in Research, Teaching & Learning and Center for the Integration of Research, Teaching & Learning (CIRTL) Network  

In this blog post, we highlight how high-impact, systems-thinking professional development practices address educational outcome inequities more effectively than other approaches, drawing from our 2015 Change article, “Faculty Development to Address the Achievement Gap”.

Institutions of higher education and K-12 school systems consist of a myriad of complex institutional, structural, organizational, and human factors that contribute to disparities in educational outcomes between minority and non-minority students. Often, however, programs, policies, and practices made to address these factors favor a “silver bullet” solution that addresses only one or two facets of the overall system, and such approaches often fail to be as effective as hoped.

A more effective path to address systemic educational inequities is to use and implement systems thinking approaches. Systems thinking integrates the various factors - and their relationships to one another - to identify, and ultimately re-design the systems that lead to poor, inadequate, and unjust outcomes.

We applied a systems thinking approach to a faculty and course instructor development and training program at the University of Wisconsin-Madison, which aimed to build faculty members’ capacity to close opportunity gaps in Science, Technology, Engineering and Math (STEM) classes.

Through a year-long, immersive program, we built course instructors’ capacity for successfully learning and implementing culturally responsive teaching practices, adapting the learning environment to align with these practices, improving engagement with students, building self-efficacy for positive change, and working effectively with student level data to identify areas for change. We focused on these instructional leaders because they are a key leverage point for implementing complex change in the higher educational system, since educator quality is the single largest factor that affects student learning that is under the control of the institution (Bensimon, 2007).

The key systems thinking levels of the systems thinking development program are summarized in the following figure: individual (faculty, course instructor), data, institutional, student, and class structure. This figure summarizes the key levels in the university system that contribute to systemic opportunity gaps. We place the faculty or instructor at the center of the system because our professional learning system leveraged the various levels of the system to build the capacity of the instructor ultimately change and improve the learning environment of the course.

image1.png
  • Individual Faculty or Course Instructor Level: Simultaneous with understanding the patterns of achievement in their classrooms, faculty immersed themselves in a year-long cohort experience that examined a range of equity issues that challenged their racial assumptions. This included group readings, activities, and speakers that educated the faculty on the impacts on student learning of microaggressions, stereotype threat, and white privilege, among other issues

  • Data Level: data were collected at various times throughout the course semester; distinctions were made between formative data (e.g., absences, tests, quizzes) and summative (e.g., drop counts/rates, final grades). These data were further stratified by gender, first generation status, minority (non-white) students, and other types of differentiating factors, such as specific discussion groups or laboratory. Faculty and course instructors did this both for their current course, and also past courses, to understand patterns and trends of when various students groups. Faculty and course instructors worked with the campus Registrar’s Office and Academic Planning and Institutional Research Office to obtain summative data across students across demographics, which is a key office to engage in this type of systems-level work to understand broader campus patterns.

    • Analysis revealed patterns in the data about when and how student performance across student groups started to decline - for example, during the first set of mid-term tests, or when particularly challenging homework was assigned.

  • Institutional Level: Faculty engaged at the institutional level with the Registrar’s Office, Academic Planning and Institutional Research, and Chief Diversity Officer Office, to understand broader patterns of opportunity gaps campus-wide, as well as other available resources and programs to support students. The Registrar’s and Academic Planning and Institutional Research Offices, in particular, are key to participants’ understanding of how their individual course data patterns fit within the patterns in courses across campus (see data-level).

  • Student Level: Faculty learned directly from students’ experiences during facilitated student panels, and were encouraged to hold focus groups with students in their classes to better understand why and how students’ failed to thrive in their courses.

  • Class Structure Level: Faculty learned about research-proven practices that are shown to positively impact opportunity gaps, such as active learning, group-based learning, creating supportive learning environments, and providing inclusive examples of diversity that are embedded in course content.

We found engaging in faculty development that embodied a systems approach to influencing classroom-based changes highly effective at positively influencing opportunity gaps. We believe that this is because the design of the professional development experience reflected the systemic nature of why opportunity gaps exist in the first place. We propose that by strengthening faculty capacity to understand their own role in addressing systemic opportunity gaps and concurrently expanding their view of how institutional, student, and teaching practice factors also contribute to opportunity gaps, developers can help faculty more intelligently adapt their own teaching practice, classroom presence, and classroom environment based on patterns they see in their own course data to positively impact student learning outcomes.
 

References

Bensimon, E.M. (2007). The underestimated significance of practitioner knowledge in the scholarship on student success. The Review of Higher Education, 30 (4), 441-469.

District Organizational Re-Design and Human Capital Management Systems

rawpixel-684806-unsplash.jpg

Districts engaging in human capital management system (HCMS) design often discover that, through the process of aligning their current educator support systems to a strategic HCMS approach, the organizational design of their districts does not support – and can even sometimes hinder – the effectiveness and benefits of a coherent HCMS strategy.

Districts’ organizational design is typified by “silos” of departments lead by district administrators that enact a top-down approach of leadership. Departments within a district lack strong, collaborative organizational “links” (such as shared, co-ownership of district priorities, projects, and programs). For example, one department may “own” a function in a district, yet not coordinate with other departments who play key roles in the management and implementation of priorities and programs.

Districts may want to consider organizational approaches that organizes district work by function, is aligned to their strategic vision, and embodies a team-based (rather than top-down) collaborative approach.

A strategic approach to HCMS in a district emphasizes alignment of management practices that support educator quality, and the districts’ organizational design should reflect and support that strategic approach. For example, Teacher Incentive Fund Program grantees have used alignment in HCMS to redesign their districts to support strategic decision making and enhance organizational coherency; please see the full report here.

 

Dr. Sara Kraemer, Ph.D, is the founder and owner of Blueprint for Education, a women-owned firm that provides  technical assistance, program evaluation, and strategic consulting in K-12 and higher education. Blueprint for Education works with clients to provide meaningful reflection and clarity on their areas of focus, while delivering the highest technical quality and analysis. Sara works closely with practitioners, researchers, policymakers, educational organizations, and institutions to achieve programmatic and organizational success.  Blueprint for Education approach is highly collaborative, needs-driven, and customizable to fit clients’ unique project goals and organizational contexts. Sara can be reached at sara@blueprintforeducation.org

How Teacher Career Ladder Positions Align to a District’s Human Capital Management Approach

rawpixel-660721-unsplash.jpg

TEACHER CAREER LADDER POSITIONS AS "HUMAN CAPITAL MANAGERS”

A significant challenge to implementing a successful comprehensive human capital management system is ensuring that that teaching and leading practices in classrooms remain focused and aligned to the broader districts goals. A critical role in this linkage are teacher career ladder positions – roles that focus on district strategic goals through aligned focus areas that relevant for new and career teacher improvement.

Teacher career ladder positions offer alternative pathways to career progression and advancement, while staying focused on teaching practice and in some cases, remaining at least part-time in the classroom. Career ladder positions, such as coaches, mentor teachers, or teacher support colleagues, serve as the connecting point between supporting teachers in the classrooms while cognizant of broader district strategic goals. Job-embedded professional supports are informed by teachers’ practice evaluations – and the evaluation system provides the unifying link between the types of professional supports needed, and overall classroom, school, and district performance goals.  

District strategic goals that focus on enhancing and supporting teacher quality may use a ‘human capital management system approach’ to align their strategic teaching and learning goals, and set the direction for schools to adapt teaching, learning, and leadership practices. Human capital management practices include teacher recruitment, hiring and selection, placement in schools and classrooms, job-embedded professional development, career ladder positions, compensation, and promotion. These practices work together as a system of supports for the arc of a teacher’s career, as well as provide a coherent set of district strategies aimed at increasing the quality of teaching and learning overall.

In the Teacher Incentive Fund grant program, grantees implement human capital management systems over the course of 5 years. One grantee implemented an innovative career ladder position – the Teacher Support Colleague – to serve as a critical link to job-embedded professional development informed by the teachers’ unique needs informed in large part by their teaching practice evaluations. Teacher Support Colleagues – who themselves are high quality, veteran teachers - serve as ‘human capital managers’ in schools. They track and support teacher professional learning needs in classrooms, align their work to the district’s broader human capital management strategy, and support school leadership decision-making around teaching practice. A unique characteristic of this position is that they are not involved in the evaluation of teachers, thus centering their role in professional supports that allow for sensitive conversations with teachers about how to improve their practice without fear of reprisal through the accountability system.

For more in-depth information of the alignment between career ladder positions and human capital management systems via the Teacher Incentive Fundplease see this report.