There are several AI based adaptive learning environments out there like Squirrel, KNewton or MathAcademy.
Are there any studies which try to extract from the (big) data of those learning systems implications of how human to human math teaching may be (significantly) improved? If so, what are the results so far?
I'm the Director of Analytics & Algorithms for Math Academy; of particular relevance to this question, I write all our adaptive learning algorithms, manage our student learning analytics, and calibrate our system to student learning data.
Yes, there are plenty of ways that human-to-human teaching can be significantly improved.
But these have been known in the literature for a long time -- for instance, mastery learning, spaced repetition, the testing effect, and varied practice have been researched extensively since the early to mid 1900s, with key findings being successfully reproduced over and over again since then.
The problem is that using these strategies systematically to their fullest extent requires an infeasible amount of effort for any human teacher.
For instance, consider spaced repetition. You have to keep track of a repetition schedule for every topic for every student, and each time a student learns (or reviews) an advanced topic, they're implicitly reviewing many simpler topics, all of whose repetition schedules need to be adjusted as a result.
In fact, before building our online system, we actually did a very loose approximation of spaced repetition while teaching in a human-to-human classroom. It turned out that, teaching just two classes with only a handful of students in each class, it took more time and effort than a full-time job to implement a very loose approximation of spaced repetition (for the class as a whole -- not even personalized to individual students). And that's just one of many strategies that are necessary for effective teaching!
Compared to the scenario described above, a standard teaching load consists of about 3x as many classes and 4x as many students per class. Let's say optimal teaching requires (conservatively) 5 different cognitive learning strategies to be implemented on a fully personalized level. Then, for a typical teacher, we can ballpark-estimate that optimal teaching would require the time and effort of about 3 x 4 x 5 = 60 full-time jobs. Which is totally infeasible.
Totally infeasible for a human, at least. But not for a machine. Our solution was to take all of these strategies (including plenty of others not mentioned above such as interleaving, layering, cognitive noninterference, cognitive load minimization) and build an adaptive automated online learning system that leverages them to their full extent. Our goal is for our system to emulate the decisions of a perfect human tutor who knows everything about their student and everything about math.
So, to summarize -- for Math Academy, at least, it's not about extracting insights from data that can inform human-to-human teaching, but rather about developing software that can actually leverage the learning strategies that have been known about for decades but require an infeasible amount of effort to implement in a human-to-human classroom.
(I'm happy to answer any follow-up questions anyone might have.)
Follow-Up Questions
The question was the other way around. For example it might be that you see some invariants from the data, i.e. that a certain way to present a topic works best for most students or you can categorize some aspects of individualization and learn from the data that in some aspects the individualization is significant and in other cases not relevant etc.
I realize your question was the other way around, but what I'm trying to get at is: in our experience, the most significant way to improve human-to-human teaching is to offload as much as possible to an automated system so that the single human teacher is no longer a bottleneck.
That said, we have learned a lot by monitoring where students get stuck and revising our content in those places to provide better scaffolding without lowering standards. The same goes for algorithms / individualization.
Our main takeaway is this: learning is like climbing a staircase, and students get stuck at any individual stair that is too tall for them to climb, so the smaller you make the individual stairs, the more students can climb all the way to the top.
Our specific improvements are part of a continual feedback loop, and there are too many to enumerate here, but if there is a particular topic or individualization in mind that you'd like me to comment on, I can probably say more about it.
