In this review the term neural networks always refers to “artificial neural networks”, because these were developed in order to emulate the biological neural networks of the human brain. However for simplicity the epithet “artificiaI” is omitted here. Neural networks consist of subelements, the neurons, which are connected together to form a network. The artificial neuron is supposed to model the functions of the biological nerve cell. Although there are at least five physiologically distinct types of nerve cell, we need only present one type here, since we discuss only the basic structure of a neuron; the physiological processes-and the chemical proceses that cause them cannot be examined in more detail.
The nerve’s cell body possesses a large number of branches, known as dendrites, which receive the signals and pass them on to the cell body. Here the signals are accumulated, and when a particular threshold limit has been exceeded, the neuron “fires”. An electrical excitation is transmitted across the axon. At its end each axon has contact with the dendrites of the neighboring neurons; this contact point is called the synapse. Neurons are linked with each other across these synapses. The synapses, however, also present a barrier that alters the intensity of the signal during transmission. The degree of alteration is determined by the synaptic strength. An input signal of intensity has an intensity of si after crossing synapse i of strength wi.
Each neuron has a large number of dendrites, and thus receives many signals simultaneously. These m signals combine into one collective signal. It is not yet known exactly how this net signal, termed Net, is derived from the individual signals. For the development of artificial neurons the following assumptions are made:
- The Net is a function of all those signals that arrive at the neuron within a certain time interval, and ofall the synaptic strengths.
- The function is usually defined as the sum of the signals sir which in turn is given by the product of the input signals x i (i =1,. .. m) and the synaptic strengths wi (i= 1,. . . m), now referred to as weights.
The net signal Net is, however, not yet the signal that is transmitted, because this collective value Net can be very large, and in particular, it can alsobe negative. It is especially the latter property that cannot be a good reflection of reality. A neuron may fire or not, but what is the meaning of a negative value? In order to attain a more realistic model, the value of Net is modified by a transfer function. In most cases a sigmoid function, also known as a logistic or Fermi function, is used. With this transfer function the range of values for the ouput signal OUI [Eq. (c)] is restricted to between zero and one, regardless of whether Net is large or small or negative.