Data, data, data. Through our work with Michigan K-12 schools and districts, we found schools who successfully raised achievement and closed achievement gaps focused on multiple forms of data. One MSU K-12 Outreach staff member observed: “The use of data for instructional improvement is often a rallying point for both administrators and teachers.” When leaders and staff engage in their own the data, they are able to take ownership of the story the data told. “When we put good data in front of teachers and walk them through the process of disaggregating the data, they can be key in identifying areas of concern. We have to slow down and have rich dialogues about the data and the process, as well as to monitor implementation,” And when staff members “put faces on the data,” they are able to make the leap to student-centered learning
Using data requires time, something that is in short supply for most schools. The most successful schools find creative ways to give staff the necessary time because they recognized that--as one MSU K-12 Outreach specialist noted--“the key to the using data to increase students’ academic performance lies in both individual and collaborative effort of staff members to understand, analyze and use pertinent information in designing energetic, meaningful, and effective lessons.”
Grant Chandler, Director of Professional Development for MSU MI Excel, Explains phase 2 of the Collaborative Learning Cycle.
Grant Chandler, Former Director of Professional Development for MSU MI Excel, Explains the Data Dialogue Process.
Grant Chandler, former Director of Professional Development for MSU MI Excel, explains Phase 1 of the Collaborative Learning Cycle.
“The amount of data educators must pay attention to can be overwhelming.Test scores, grade points, drop-out rates, and graduation rates all become markers of a school’s success. Amidst this inundation, however, it is not always clear how to effectively organize and analyze data.
“Numbers can tell a story,” Victoria Bernhardt said. “When we honestly see what the numbers are telling us, we can always improve upon our story.”
Victoria Bernhardt, Ph.D., is the executive director of Education for the Future Initiative and professor at California State University. As a data analyst who has worked with many districts for school improvement, Bernhardt advises schools to look at all of their data to understand the needs in a comprehensive and objective manner.
“My advice for school administrators focusing on improvement in one area is to …understand how the different data elements are related, and how they move together.”
Last July, Victoria visited Michigan State University for the Superintendents’ Summer Institute. During her visit, she outlined effective use of data for school improvement that can be tailored for each school’s needs. Victoria took the time to speak with MIToolkit about improving teaching and learning, multiple ways to look at data and the meaning behind her work.
MIToolkit: Tell us a little bit about yourself. You’re a statistician. How did you become interested in analyzing school data specifically?
Victoria Bernhardt: I love how numbers can tell a story. When we honestly see what the numbers are telling us, we can always improve upon our story. My interest in analyzing school data began with evaluating school programs. I could see how easily schools could improve their programs if they would just act upon what the data are telling them. It seemed to me that if we can gather data to improve programs, we can also gather data to improve schools.
MT: You say that 80% of what needs to change is us, not students. Could you elaborate on this?
VB:The systems thinkers, W. Edwards Deming and Peter Senge, say that 80% of the results we are getting are attributed to the processes. We need to study the impact of our processes and adjust to improve the results. Teachers need to develop the ability to reflect on how they deliver instruction and be willing to change these processes so all students are successful.
MT: Regarding the continuous school improvement framework, you express concern over schools that are planning their improvement using students’ summative learning results only. Why do you believe demographics and perceptions should come before summative student learning results when analyzing data?
VB:When schools use summative student learning results only, they tend to add solutions to “fix” the kids, such as after-school programs, tutoring programs, interventions, and the list goes on. This approach implies that what the adults are doing is fine: the kids just need more. I believe we need to take all types of data into consideration before we come up with solutions to our undesirable results.
MT: What are the strategies or preconditions that lead to effective use of student learning data?
VB:There are five preconditions for the effective use of student learning data.
- The use of appropriate data. Student learning data must predict what you want to measure students on ultimately (such as your summative assessments).
- A shared vision. A shared vision ensures that all school staff have a common understanding of why they are assessing students, what they are going to do when students are proficient, and what they are going to do when students are not proficient.
- Leadership encouragement and support. That shared vision of data use must be modeled and reinforced by leaders. Leaders must be aware of the work that is going on throughout the school, encourage the collaboration of staff and the improvement of instruction, and monitor the school-wide data.
- Structures for collaboration. Peter Senge says, “Collaboration is vital to sustain what we call profound or really deep change, because without it, organizations are just overwhelmed by the forces of the status quo.” We must create the structures for staff collaboration.
- Strategies to support each other in the attainment of new teaching skills. What teachers do in their collaborative structures matter. Schools must adopt strategies that can help teachers learn together, improve their teaching, and help all students learn. This will not happen on its own. You cannot just create the structures. You must also adopt strategies to make sure the structures can create the results you want.
MT: Why are the school processes data important to continuous school improvement and how do you analyze them?
VB:This quote, adapted from W. Edwards Deming, sums up why school processes data are important for continuous school improvement:
“Schools” are perfectly designed to get the results they are getting now. If “schools” want different results, they must measure and then change their processes to get the results they really want.
School processes are the only measures over which we have the most control in the education setting. Schools cannot control who the students are, where they come from, or why they think the way they do when they come to us. However, schools can control a major portion of the student learning results through their processes (i.e., curriculum, instructional strategies, assessment practices, programs or environment). Through these processes, schools ultimately control their outcomes.
To analyze school process data:
- List the programs and processes operating in the school.
- Analyze the lists of programs and processes for deletions and additions.
- Analyze each program and process in such a way that you can get implementation with integrity and fidelity, and see results.
MT: One of the lines that stuck with me during the Summer Institute was “If you are not monitoring and measuring program implementation, the program probably doesn’t exist.” What are some ways of monitoring the program for continuous improvement?
VB: When schools clarify what they want teachers to implement, then monitor and evaluate the implementation, they will be able to see the impact of the program/process. When schools describe what it will look like to implement a program or process, they can turn those steps into components to look for in classrooms.
MT: You are well known for tailoring the data for each school’s needs. What's your advice for a team of school administrators and educators who are trying to find their focus improvement point?
VB:I hope that I am known for saying that all schools need to look at all their data to understand their needs in a comprehensive and objective manner. My advice for school administrators focusing on improving one area is to not just focus on that area. Understand how the different data elements are related, and how they move together.