Using Rapid-Cycle Evaluation to Stop the Edtech Bloat
Post-pandemic, districts are looking to eliminate the edtech bloat and determine which products they should keep and which ones to scale back. Of course, educators and administrators could look at past success stories, research data, etc. But what about how the programs are impacting current students?
During the edLeader Panel, “Stop the Edtech Guesswork: Make Evidence-Based, Not Anecdote-Based, Decisions,” district leaders advocated for rapid-cycle evaluation to help districts understand the impact of their edtech purchases now.
According to LearnPlatform by Instructure, “Rapid-cycle evaluation (RCE) is a formative decision-making process that allows administrators to examine what works in their local contexts using real-world data and a cyclical research design conducted within a condensed time frame.”
The RCE process begins with a research question, e.g., Is product x improving outcomes for learning goal z? Then, admins identify what data is relevant to this specific question, collect the data, and analyze it.
While there are benefits to long-term analysis, the panelists explained their key reasons for choosing RCE:
- They can identify if a program is having a quantifiable impact on their students now and change course if needed.
- In addition, they can find out if a program is doing what they think it should be doing or if it’s having unintended benefits—or consequences.
- Admins can also see how many of their staff are using the program, and if they’re using it with fidelity, i.e., how the product says it needs to be used in order to attain the best results.
There are, though, some challenges when implementing RCE:
- Staff need training not only on how to gather and interpret the data but also to understand why the district is using RCE and how it will benefit them and their students.
- Vendors need to make sure their programs are providing the data the schools need, and if not, they need to adjust their programs to provide the necessary information.
- Every program cannot be evaluated at once. The panelists suggested beginning with the top three questions or programs.
In fact, all of the panelists recommended starting with the high-priced items since they should be having the biggest impact on students as well as the budget.
They’ve used the results of RCE to help them understand what students and situations generate the most impact from a product, what type of licensing they should secure for the next school year, and what gaps they have with their edtech products.
Each process will lead to more questions, but that is part of the RCE cycle. When every staff member is looking at what edtech products are working and under what conditions, teachers and administrators can have meaningful conversations about how to improve learning.
Learn more about this edWeb broadcast, “Stop the Edtech Guesswork: Make Evidence-Based, Not Anecdote-Based, Decisions,” sponsored by LearnPlatform by Instructure.
Join the Community
Technology in Schools is a free professional learning community where district administrators, school leaders, and all educators can share ideas, examples, and resources that relate to integrating technology effectively in schools.
Recently named as one of Fast Company’s World’s Most Innovative Companies in 2023, LearnPlatform by Instructure serves 10 million students and since 2015 has run over 1,200 rapid-cycle evaluations on specific edtech solutions directly with its district partners, saving millions of dollars and increasing the speed of decision-making by more than 85 percent. LearnPlatform’s innovative edtech effectiveness system simplifies and optimizes systems and processes for K-12 educators and administrators to find, select and evaluate edtech tools to ensure they are safe, compliant with privacy requirements, equitable, and delivering academic and financial return on their investment. For edtech providers of any size, LearnPlatform also offers subscription-based research services to meet ESSA compliance and build evidence-based interventions that empower districts and states to buy and use their products to drive student outcomes.
Blog post by Stacey Pusey, based on this edLeader Panel.