The first neural network I wrote was a neural network called the Hopfield Network. It is a neural network that can memorize patterns and recall them. The way the network does this is by saving the patterns in a matrix, and then recalling it by multiplying the pattern to recall by the matrix. This network was the first neural network created, and was created by John Hopfield. This network relies on binary inputs which are fed into a neural network that is completely interconnected, (figure 1) these interconnections are known as weights, and the connection between each pair of input units is the value of each weights. The equation for the weights connecting two input units is given in figure 2. Once you have this weight matrix, you get outputs by recursively multiplying your input by the weights matrix. E.g, if you have a weight matrix W, and you have an input matrix I that the network hasn't ever seen before, you can get the "closest" result by multiply W*I, taking that result, and multiplying by W again and again until the pattern converges.
Code is at GitHub at:
https://github.com/elihulse5/Hopfield-Network
Figure 1:
Figure 2:
Some sample recall values:
