Tay Moss

Can AI Help Every Student Thrive? What the 'Two Sigma Problem' Tells Us About the Future of Learning 

“I believe that an important task of research and instruction is to seek ways of accomplishing this under more practical and realistic conditions than the one-to-one tutoring, which is too costly for most societies to bear on a large scale. This is the '2 sigma' problem." (Bloom in Original Article )

In 1984, educational researcher Benjamin Bloom (the creator of the “Bloom Taxonomy” which I've written about before) made a discovery that rocked the education world. He found that students who received one-on-one tutoring performed two standard deviations better than those in a traditional classroom. That means the average tutored student outperformed 98% of students taught the usual way. This became known as the "Two Sigma Problem." The question is: How can we give every student that level of personalized learning without needing a personal tutor for each one? 

At CHURCHx, we're diving deep into this question because it’s more than academic—it’s pastoral. As pastors, church educators, and leaders, we care about people growing in wisdom and faith. But many of our learners—especially adult learners—struggle with time, energy, and confidence. If there's a way to help more of them learn and grow better, faster, and with more joy, we want to know about it. 

Why Tutoring Works So Well 

Tutoring isn't just about getting answers—it's about teaching in a way that adapts to the student. A good tutor offers timely feedback, adjusts the pace, encourages questions, and helps the student stay motivated. Bloom showed that this kind of support can turn average learners into top performers. But giving every student their own personal tutor? That's simply not practical outside of very specific circumstances (such as field placements for ministry students). 

So for decades, educators have tried to replicate the benefits of tutoring in the classroom—through things like mastery learning, small group work, and peer tutoring. These methods help, but none have consistently achieved the two-sigma boost that tutoring provides.  

Enter AI: A New Kind of Helper 

Students that worked with AI tutors reported feeling more engaged and motivated. ( Original Article )

Thanks to recent advances in artificial intelligence (AI), we now have tools that can simulate some of what a tutor does. At Harvard University, researchers created an AI-powered tutor and tested it against a traditional classroom. The results? Students using the AI tutor learned twice as much in less time—and reported feeling more engaged and motivated, too. 

This AI wasn’t just a chatbot with answers. It was carefully designed to teach like a great tutor: asking questions, encouraging effort, giving feedback, and letting students work at their own pace. The key was human-centered design—experts crafted the AI to follow best practices from teaching and psychology. 

While it’s exciting to imagine every student in the world having access to a Harvard-AI-Tutor-like bot, two important caveats need to be expressed. First, the Harvard-bot was very carefully and deliberately designed to use a specific learning path designed by the instructors over many successive iterations of the physics class it was designed to emulate. It was not a case where the instructors simply “dumped” the learning outcomes into ChatGPT and said, “Teach this.” Rather, this is AI applying a human-designed strategy with great fidelity and at scale. 

Second, the students taking this class are likely to be highly motivated and self-directed learners, already, and therefore these gains might be unusual when extrapolated over a more diverse community of learners. That said, although the Harvard study is a particularly shocking result, it not unusual for the current (as of Fall 2024/Spring 2205) generation-based AI-Tutors to demonstrate great benefits for students. Multiple studies have shown that with the right prompting, AI makes an excellent tutor. Some of the most fruitful research on this topic is about how to prompt the models to get best results. 

But AI Isn’t Magic 

Here’s the catch: AI only works well when it’s used wisely. Some tools can accidentally make it too easy for students to avoid real learning. Others might give wrong answers or lack the emotional support that human teachers provide. That’s why researchers and educators are emphasizing a partnership between humans and AI. The goal isn’t to replace teachers, but to amplify their work—letting the AI handle the repetitive stuff so teachers can focus on mentoring, spiritual formation, and community-building.  

None of this is suggesting a shortcut to teaching or learning. Unlike in the movie The Matrix, we can’t plug brains into machines and “upload” understanding. Indeed, the proper construction of online learning (whether using AI-based learning activities or not) often requires even more planning and resource creation with the payoff of better engagement and outcomes. 

As I tell instructors sometimes, “Students must suffer!” That is, a certain effort is required to learn: learning is a kind of gym of social, mental, and sometimes physical training that changes the human person as they adapt to the carefully designed exercises put before them. A critical choice in designing any learning experience is where in the design this student-lift will take place. AI is just another tool to create those learning activities. 

Why This Matters for CHURCHx 

At CHURCHx, we see AI as a powerful tool for discipleship and lifelong learning. Many groups want to offer deeper, more personalized learning but don’t have the staff or resources to provide individual support. What if AI tutors could help bridge that gap? Imagine: 

• A confirmation student getting instant help with Bible questions at home. 
• A lay leader using an AI tutor to explore theology at their own pace. 
• A pastor preparing a sermon with the support of an AI tool trained on trusted resources. 
• We’re exploring how to build and use AI tutors that are trained on church-based content, guided by our values, and supportive of our mission. We want tools that encourage critical thinking, promote spiritual growth, and respect the learner’s dignity.