New research from Leanlab highlights barriers to AI adoption in education

The growing field of AI-powered educational technology is undergoing rigorous real-world testing to bridge the gap between developer intentions and classroom realities.

Research methodology and background: Leanlab Education, a Kansas-based nonprofit, has been conducting “codesign research” to evaluate emerging educational technologies through direct classroom implementation and feedback.

• The organization has a decade-long history of matching school districts with ed-tech developers for field research
• Recent studies have shifted focus to AI tools in response to their rapid proliferation in education
• Between January and August, researchers examined five AI-powered ed-tech tools across six classroom studies

Key expectations and challenges: Initial alignment between educator needs and developer goals revealed significant technical and practical hurdles in real-world implementation.

• Teachers sought tools to save time, support differentiated instruction, and enhance student engagement
• Technical issues frequently disrupted smooth implementation of the tools
• Integration with existing teacher workflows proved problematic
• Educator trust emerged as a significant barrier due to concerns about AI output accuracy and quality

Performance issues and content limitations: Real-world testing revealed significant operational inefficiencies and content shortcomings that impacted tool effectiveness.

• Some tools required up to an hour to generate lesson plans, making them impractical for daily use
• Student-focused content generation was both slow and insufficiently complex to maintain engagement
• Differentiation capabilities failed to adequately support lower-skill students
• Tools often assumed baseline proficiency levels that excluded struggling learners

Developer response and iterations: The research process enabled real-time improvements based on educator feedback.

• Developers addressed content loading times and complexity issues during the study period
• Recommendations included adding scaffolding for struggling students
• Suggested improvements included customizable audio/video playback speeds and adjustable reading levels
• The rapid-cycle evaluation process allowed for quick implementation of user-suggested changes

Trust building and future implications: The path to widespread AI tool adoption in education requires establishing educator confidence through demonstrated reliability and effectiveness.

• Each instance of unpredictable output reinforces teacher skepticism about AI tools
• Real-world testing creates opportunities to build trust through responsive development
• Operational stability must precede rigorous impact studies
• Future research will focus on quantitative studies to measure actual educational outcomes

Looking ahead: The success of AI-powered educational tools will depend largely on developers’ ability to address fundamental usability issues while building educator trust through consistent, reliable performance. The feedback loop between classroom experience and tool development appears crucial for creating truly effective educational AI solutions.