Introduction to Neural Networks

Author: Peter Bradley

Neural networks models — often called parallel distributed processing models or connectionist models — begin with the assumption that natural cognition takes place through the interactions of large numbers of simple processing units. Inspiration for this approach comes from the fact that the brain appears to consist of vast numbers of such units — neurons. While connectionists often seek to capture putative principles of neural computation in their models, the units in an actual connectionist simulation model should not generally be thought of as corresponding to individual neurons, because there are far fewer units in most simulations than neurons in the relevant brain regions, and because some of the properties of the units used may not be exactly neuron-like.

Features of a Neural Network model

The "knowledge" that governs processing in a connectionist model consists of the values of the connection weights. These weights are changed through the process of 'learning', or the gradual adaptation of the connection weights over a period of training. For the sake of simplicity, we'll start with networks where all the connection weights are simply 1.

Introductory Neural Network

Each little black circle is a unit or 'node'. The number on that node is it's threshold. When a node turns red, it is active and produces an output of '1'. In this simple demonstration, each connection has a weight of 1, so every connection leading from an active node has a value of 1; every connection leading from an inactive node has a value of 0. In order to determine the level of activity on a given node, we add up the values of all the connections coming into that node. If that result is greater than the node's threshold, that node becomes active.

To use the network above, first set the thresholds of the nodes, and then click 'Run'. At this point, you can click on any of the nodes to activate them. Once you have activated your desired nodes, click on 'Step'. The network will run through one cycle, propagating the activity forward through the network. For each node, the system calculates the net input to a node, compares it to the threshold of that nodes, and activates it if necessary. Press 'Step' again to initiate another cycle.

Copyright: 2007