Return to MODULE PAGE
McCulloch-Pitts Neurons: Introductory Level (page 4)
Michael Marsalli: Author
Beyond the McCulloch-Pitts neuron
So far if we wanted to find an MCP neuron that had a certain type of behavior, we had to guess the combination of signals and threshold that would work. We are now going to see how an MCP neuron can ``learn" how to produce the type of behavior we want. But first we need to make our MCP neurons a bit more flexible. What we are going to do was not done by McCulloch and Pitts. But it is an important step in the development of modern neural networks.
Suppose we have an MCP neuron connected to two detectors. We'll call the first detector X1 and the second detector X2 for short. We won't be concerned with exactly what the detectors are detecting. For shorthand we will write a 1 if the detector sends a signal to the neuron, and we will write a 0 if the detector sends no signal. Now there are four possible situations for the two detectors.
Both detectors can send a signal, in which case we have X1 is 1 and X2 is 1.
The first detector sends a signal, but the second detector sends no signal, in which case X1 is 1 and X2 is 0.
The first detector sends no signal, but the second detector does send a signal, in which case X1 is 0 and X2 is 1.
Finally, neither detector sends a signal, in which case X1 is 0 and X2 is 0.
We can summarize these four possibilities in the following table.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
We now want a way to take into account the possibility that one of our detectors has more influence on the MCP neuron than the other detector. In other words, it's possible that the signal from one detector may be much stronger than the other signal. The way we will do this is by assigning a strength, or weight, to each detector. The weight of a detector will tell us how much influence the detector's signal has on the MCP neuron.
The weight will be a decimal number, and it can be positive or negative. A positive weight will mean that the signal is excitatory, and a negative weight will mean that the signal is inhibitory. We'll use W1 as shorthand for the weight of the first detector, and we'll use W2 as shorthand for the weight of the second detector. We'll also need a threshold, just as before. The threshold will also be a decimal number. Now we'll see how the MCP neuron uses these weights.
Recall from the previous lessons that we added the incoming signals and compared the total to the threshold. We will still do this, except that instead of using a 1 for an incoming signal from a detector, we will use the weight for that detector. If no signal is sent, we will still use a 0. Let's look at an example.
Suppose that the weight W1 is 0.5, the weight W2 is -0.3, and the threshold is 0.1. These are just convenient choices for the weights and threshold, we could have made other choices. Let's look at how this particular MCP neuron would work in each of the four possible combinations for the two detectors.
If both detectors send a signal, i.e. X1 is 1 and X2 is 1, then we would take the weight for the first detector and the weight for the second detector and add them together. In this case, the weight of the first detector is 0.5 and the weight of the second detector is -0.3, so when we add these we get 0.5 + (-0.3) = 0.2. Now we compare this total to the threshold 0.1. Because the total 0.2 is greater than the threshold 0.1, the MCP neuron will send out a signal 1.
If the first detector sends a signal, but the second detector sends no signal, i.e. X1 is 1 and X2 is 0, then we just use the weight from the first detector, and we use a 0 for the second detector. Since the weight for the first detector is 0.5, when we add the signals we get 0.5 + 0 = 0.5. Now we compare this total to the threshold 0.1. Because the total 0.5 is greater than the threshold 0.1, the MCP neuron will send out a signal 1.
If the first detector sends no signal, but the second detector does send a signal, i.e. X1 is 0 and X2 is 1, then we use a 0 for the first detector, and we use the weight for the second detector. Since the weight for the second detector is -0.3, when we add the signals we get 0 + (-0.3) = -0.3. Now we compare this total to the threshold 0.1. Because the total -0.3 is less than the threshold 0.1, the MCP neuron will send out a 0.
Finally, if neither detector sends a signal, i.e. X1 is 0 and X2 is 0, then we use a 0 for both detectors. So the total is 0 + 0 = 0. Again we compare the total to the threshold 0.1. Because the total 0 is less than the threshold 0.1, the MCP neuron will send out a 0.
So if X1 is 1 and X2 is 1, then the MCP neuron produces a 1. If X1 is 1 and X2 is 0, then the MCP neuron produces a 1. If X1 is 0 and X2 is 1, the MCP neuron produces a 0. Finally, if X1 is 0 and X2 is 0, then the MCP neuron produces a 0. We can briefly summarize all this information in the following table, where we use an F as shorthand for the output of the MCP neuron.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Fortunately, you won't have to do all the above calcualtions for any other MCP neurons, because we have a program that can produce the above table when you enter the weights and threshold. Just enter the weight for the first detector in the space next to W1, the weight for the second detector in the space next to W2, and the threshold in the space next to T. After you enter the weights and threshold, click on the "Run Simulation" button, and the program will then produce a table like the one above. To start the program click on the following button.
Exercise. Use the program above to produce the table for an MCP neuron where W1 is 0.5, W2 is 0.5, and the threshold is 1.
Now that we have MCP neurons with weights, we will show how an MCP neuron can ``learn" to produce the behavior we want. This was the next big step in the development of the modern neural network, and the basic idea was due to the psycologist Donald Hebb in 1949. Hebb was trying to explain how we learn in terms of the behavior of our neurons.
He thought that if a detector repeatedly and persistently sent a signal to a neuron, thus exciting the neuron to send out its own signal, then somehow the connection from the detector to the neuron would be strengthened. We will translate this idea into a ``learning" rule for our MCP neuron with weights.
Let's take look back at our very first example. We have a bird with a neuron connected to two detectors. The first detector sends a signal if the object is round, and the second detector sends a signal if the object is purple. We want the neuron to send a signal only when an object is both round and purple, so that the bird will only eat round, purple objects.
Our neuron now has a threshold and two weights, one weight for each detector. Here's how we'll use Hebb's idea to ``train" our neuron to produce the correct signals. If the object is round and purple, and the neuron is sending a signal, we won't change anything. If the object is round and purple, and the neuron is not sending a signal, we'll add 1 to the weights on the detectors, and we'll subtract 1 from the threshold. If the object is not both round and purple, and the neuron is sending a signal, then we'll subtract 1 the weights of the detectors that are also sending a signal, and we'll add 1 to the threshold. If the object is not both round and purple, and the neuron is not sending a signal, then we won't change anything.
Notice that when the neuron has the correct behavior, we don't change anything. But when the neuron doesn't have the correct behavior, we try to change the weights and threshold in such a way as to help the neuron to produce the correct behavior.
When the neuron should be sending a signal, but isn't, we increase the weights of the detectors that are sending a signal, and we lower the threshold. This makes the total of the weights larger and the threshold smaller, so the total is closer to exceeding the threshold. Thus the neuron is either now sending a signal, or it's getting closer to sending a signal.
When the neuron is sending a signal, but shouldn't, we lower the weights of the detectors that are sending a signal, and we increase the threshold. This makes the total of the weights smaller and the threshold larger, so the total is farther from exceeding the threshold. Thus the neuron is either no longer sending a signal, or it's getting closer to not sending a signal.
Unfortunately, one adjustment of weights may not be enough to produce the behavior we want under all circumstances. We may have to repeat the adjustment of weights many times for all the circumstances until the neuron always produces the correct behavior in every circumstance. This is a lot like ``learning." When we learn some task, we often have to practice many times until we get it right.
Now that we have seen how to take an individual MCP neuron and "train" it to perform a desired behavior, we are ready to consider the next development in the history of neural networks: the perceptron.
