What is a good teaching example of machine learning?

Question

I'll be giving a guest lecture in a grade 12 calculus class. I'm going to give them an introduction to machine learning, and go through some of the math involved in training an adaline. It would be helpful to have some simple examples of problems that could be solved with such a technique.

Give an example, either practical or fun, of a binary decision that depends on one or two real valued variables.

For example, an email filter might mark as spam a message which contains the words nigerian and prince.

Answer 1

I'm not sure that neural network is the best example of machine learning, or that spam filters are good example of neural network usage. In particular, a decision that depends on "one or two real valued variables" seem neither practical nor fun. You can easily create a more sophisticated network, but there are still some issues.

Discrete things like "contains words $X$" cause problems because you would need to have a separate input for each (common) word indicating if it is in the message. If you insist, you can find some info about spam filter networks in this question.

Also neural networks are good only in some range of applications, and so it happens that humans are also quite good in number of them (guess why...), so that neural network machine learning doesn't seem that impressive (e.g. random forests give nice results and are easy to understand, while Bayes-based classifiers may have some educational advantage).

That being said, I would recommend anything based on pattern matching, e.g.

• approximating a function, a visualization how the network learns is often engaging, there are tons of examples online;
• character recognition (simple OCR), the simplest feature set is take a random set of short lines, if you would like to use something closer to human visual system, you can use Gabor stimuli;
• voice recognition, I've never did this, but it seems possible, you could record the voices of your students and then train the network to recognize them, you could also try to guess the gender of the speaker (e.g. based on the histogram).

Also, to indicate how the neural networks works, you could do some weird things, like guessing the color of the hair based on voice. Then explain that it's reasonable to suspect there is no correlation between the two things in general (esp. if some girls use hair dye), but the classifier learned the voices of the people in class and approximately who has which color, it would fail on someone out of the training set.

I hope this helps $\ddot\smile$

Answer 2

Let me reiterate Roland's and dtidarek's suggestion to use character recognition, and specifically numeral recognition, as a teaching example. There are three advantages to this example:

1. It is easy to understand.
2. It is the basis of zipcode recognition: It is being used every day, implemented in hardware used by the US Postoffice.
3. There are simulators all over the web to explore performance. E.g., Hinton's.

Here is a seminal paper on the topic that has subsequently been cited more than 1000 times:

LeCun, Yann, Bernhard Boser, John S. Denker, Donnie Henderson, Richard E. Howard, Wayne Hubbard, and Lawrence D. Jackel. "Backpropagation applied to handwritten zip code recognition." Neural Computation 1, no. 4 (1989): 541-551.

---

     _{(Part of) Fig. 1.}

---

Answer 3

Coursera's Machine Learning Course by Andrew Ng used two examples to motivate the training of neural networks:

1. Programming of Logic gates, such as AND, OR, XNOR and so on - the latter requires two layers.

2. Number/character recognition. This was demonstrated with a video from Yann LeCuns NCR for AT&T from the 1990's as well as a programming exercise.