Enhancing mathematical proof skills using AI (in university teaching)

Question

I would appreciate ideas on good practice in the use of AI among mathematics and statistics students to improve their critical thinking in the development of mathematical proofs.

I am wondering about students:

1. sourcing a proof of a particular theorem, or a property of a parameter, and then being asked to critique the proof;

2. developing a different proof to that already provided using generative AI prompts;

or

3. a combination of 1) and 2), above.

Are any AI tools particularly helpful for this sort of thing? Would using them require payment of a fee by the institution or students?

Answer 1

Unfortunately I can only offer two warnings.

1. In the article of Krupp et al. (2024) they find the various students that study physics as a part of their education, when faced with difficult physics tasks, trust blindly on what Chat-GPT tells them. The ones who use it do a lot worse than the ones who have access to search engines and internet.

2. The classical article of Selden and Selden (2003) suggests that bachelor students have great difficulties distinguishing correct proofs from false ones and focus on the surface features of the proofs, rather than the logic in them.

3. The way large language models (LLMs) in particular work is that they repeat the surface structures they have seen in a lot of other texts. If they repeat the surface features of a mathematical proof well enough, it might even be a valid proof! There is a lot training material available, at least, so this is possible. However, an LLM might just as well write something that resembles a proof but is not it.

As per one and two above, students will have hard time noticing these false proofs and might even trust them without further reflection.

This does suggest a fruitful avenue for using LLMs: have them generate a large number of proof candidates, correct and false, and then have students figure which is which.

You would have to get familiar with a particular LLM to know which capabilities it has in proof writing. There is plenty of research on the mathematical capabilities of at least Chat-GPT and on the use of artificial intelligence in mathematics education in general, so maybe read a review article or two and see what you find there. This is currently a hot topic. (Unfortunately also in terms of climate effects, not just scientific interest...)

References

KRUPP, Lars, et al. Unreflected acceptance–investigating the negative consequences of chatgpt-assisted problem solving in physics education. In: HHAI 2024: Hybrid Human AI Systems for the Social Good. IOS Press, 2024. p. 199-212.

SELDEN, Annie; SELDEN, John. Validations of proofs considered as texts: Can undergraduates tell whether an argument proves a theorem?. Journal for research in mathematics education, 2003, 34.1: 4-36.