A multi-layer feed-forward neural network, MFNN, is a network composed of multiple layers each of which contains a number of neurons. The first layer is called the input layer. Neurons in the input layer are usually simply signal or features input points, i.e. input neurons do not process input signals, they only provide the signals to the next layer. The next layer after the input layer is called a hidden layer. There can be an arbitrary number of hidden layer each containing an arbitrary number of neurons. Nonetheless, most models contain three layers: 1) an input layer, 2) a hidden layer and 3) an output layer. The output layer is the corresponding reverse layer to the input layer, i.e. it is the layer where output neurons output their values. In contrast to the input layer, output neurons process their incoming signals. Hidden layers are what gives the network the ability to model arbitrary nonlinear functions. There are, generally, no rules to set the number of layers in a network and the number of nodes in each layer for a specific task. Cross-validation, i.e. changing these parameters whilst tracing the end result, and genetic algorithms can be used to heuristically determine the number of layers and neurons.
The qualification 'feed-forward' means that signals flow from a layer to the next only in the forward direction starting at the input layer. Niether loops nor feeding a signal backwards take place in a feed-forward network. In contrast to feed-forward networks recurrent networks exhibit loops and feeding signals backwards.
MFNNs are able to solve a wide array of problems. To begin with, as discussed previously, MFNNs can model arbitrary mathematical functions. They can be used to implement character and text recognition systems, image classification systems (although convolutional neural networks may be easier to devise in this case), regression problems solutions, etc.
It may be evident by now that implementing a good network requires a non-trivial thoughtful process and a recognisable amount of work. In fact, this pertains to most of distributed systems in general, if you would consider neural networks to be distributed systems. So how should we go about implementing the multi-layer XOR network we previously devised? There are a number of elements to consider: neurons and their activation functions, weights and signal flow. It turns out that implementing the network in the crudest form gives good clues about the process. The following section discusses is a crude implementation of the XOR MFNN in a single function.