8
$\begingroup$

I'll be giving a guest lecture in a grade 12 calculus class. I'm going to give them an introduction to machine learning, and go through some of the math involved in training an adaline. It would be helpful to have some simple examples of problems that could be solved with such a technique.

Give an example, either practical or fun, of a binary decision that depends on one or two real valued variables.

For example, an email filter might mark as spam a message which contains the words nigerian and prince.

$\endgroup$
  • $\begingroup$ (Sorry, but I couldn't resist.) Point out that the (Arnold) Terminator in the 1991 movie Terminator 2 said "My CPU is a neural net processor; a learning computer", and then discuss the relevance of the Terminator later saying "Hasta la vista, baby". $\endgroup$ – Dave L Renfro Jun 10 '14 at 16:06
  • $\begingroup$ Of interest: Neural Networks and Deep Learning In self learning I found this to be the best and simplest one to get started. $\endgroup$ – Guy Coder Nov 24 '16 at 20:31
4
$\begingroup$

I'm not sure that neural network is the best example of machine learning, or that spam filters are good example of neural network usage. In particular, a decision that depends on "one or two real valued variables" seem neither practical nor fun. You can easily create a more sophisticated network, but there are still some issues.

Discrete things like "contains words $X$" cause problems because you would need to have a separate input for each (common) word indicating if it is in the message. If you insist, you can find some info about spam filter networks in this question.

Also neural networks are good only in some range of applications, and so it happens that humans are also quite good in number of them (guess why...), so that neural network machine learning doesn't seem that impressive (e.g. random forests give nice results and are easy to understand, while Bayes-based classifiers may have some educational advantage).

That being said, I would recommend anything based on pattern matching, e.g.

  • approximating a function, a visualization how the network learns is often engaging, there are tons of examples online;
  • character recognition (simple OCR), the simplest feature set is take a random set of short lines, if you would like to use something closer to human visual system, you can use Gabor stimuli;
  • voice recognition, I've never did this, but it seems possible, you could record the voices of your students and then train the network to recognize them, you could also try to guess the gender of the speaker (e.g. based on the histogram).

Also, to indicate how the neural networks works, you could do some weird things, like guessing the color of the hair based on voice. Then explain that it's reasonable to suspect there is no correlation between the two things in general (esp. if some girls use hair dye), but the classifier learned the voices of the people in class and approximately who has which color, it would fail on someone out of the training set.

I hope this helps $\ddot\smile$

$\endgroup$
3
$\begingroup$

Let me reiterate Roland's and dtidarek's suggestion to use character recognition, and specifically numeral recognition, as a teaching example. There are three advantages to this example:

  1. It is easy to understand.
  2. It is the basis of zipcode recognition: It is being used every day, implemented in hardware used by the US Postoffice.
  3. There are simulators all over the web to explore performance. E.g., Hinton's.

Here is a seminal paper on the topic that has subsequently been cited more than 1000 times:

LeCun, Yann, Bernhard Boser, John S. Denker, Donnie Henderson, Richard E. Howard, Wayne Hubbard, and Lawrence D. Jackel. "Backpropagation applied to handwritten zip code recognition." Neural Computation 1, no. 4 (1989): 541-551.


Fig1
     (Part of) Fig. 1.


$\endgroup$
  • 1
    $\begingroup$ I think this is what I'm going to go with. $\endgroup$ – Anthony Jun 15 '14 at 0:02
1
$\begingroup$

Coursera's Machine Learning Course by Andrew Ng used two examples to motivate the training of neural networks:

  1. Programming of Logic gates, such as AND, OR, XNOR and so on - the latter requires two layers.

  2. Number/character recognition. This was demonstrated with a video from Yann LeCuns NCR for AT&T from the 1990's as well as a programming exercise.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.