Radial Basis Function Networks - Universal Function Approximation
A Radial Basis Function (RBF) network [1,2] is a two-layer neural network, the output units of
which form a linear combination of the basis functions computed by the hidden units, φj. The activation of the
hidden units in an RBF network is determined by the Euclidean distance between the input x and a set of prototype vectors.
A typical RBF network is shown in the figure to the left.
The first layer's nonlinear mapping is constructed using a set of basis functions whose centres correspond to the prototype vectors in input space. These basis functions are usually chosen to be Gaussians (bell-shaped functions) and the output activity of the jth hidden unit is then given by a linear combination of these Gaussians.
The width, or sharpness of the Gaussian for that prototype or centre is a parameter which is chosen to ensure that the basis functions give rise to a localized repsonse for any imput pattern, i.e. they only produce a significant non-zero repsonse when the input pattern belongs to a small localized region of input space. The name "radial basis function" comes from the fact that the Gaussian functions are radially symmetric: each hidden unit j produces an identical output for input patterns that are found within a fixed radial distance from the centre of the basis function [3].The Gaussian basis functions are not normalized to unity height since their amplitude is effectively set by the weights of the second layer. Thus the output layer forms a weighted linear combination fo the hidden unit activations. Hence an RBF network performs a nonlinear transformation from the input space to the output space by forming a linear combination of nonlinear basis functions.
[1] D. Broomhead, D. Lowe (1988), Multivariable Function Interpolation and Adaptive Networks. Complex Systems, 2:321-355.
[2] J. Platt (1991), A Resource-Allocating Network for Function Interpolation. Neural Computation, 3(2):213-225.
[3] L. Tarassenko (1998), A guide to neural computing applications. London, New York: Arnold.