Types Of Neural Networks

Multilayer Perceptrons

Multilayer Perceptrons, or MLPs for short, are comprised of one or more layers of neurons. Data is fed to the input layer, there may be one or more hidden layers providing levels of abstraction, and predictions are made on the output layer.

Types of problems MLPs solve:

• MLPs are suitable for classification prediction problems where inputs are assigned a class or label.

• MLPs are also suitable for regression prediction problems where a real-valued quantity is predicted given a set of inputs. Data is often provided in a tabular format, such as you would see in a CSV file or a spreadsheet.

• They are very flexible and can be used generally to learn a mapping from inputs to outputs.

• This flexibility allows them to be applied to other types of data. For example, the pixels of an image can be reduced down to a 1D array of data and fed into a MLP. The words of a document can also be reduced to a 1D array of data and fed to a MLP. Even the lag observations for a time series prediction problem can be reduced to a1D array of data and fed to a MLP.

• If your data is in a form other than a tabular dataset, such as an image, document, or time series, you can at least test a MLP on your problem. The results can be used as a baseline point of comparison to confirm that other models that may be better suited to solve the problem.

Try MLPs On:

• Image data
• Text Data
• Time series data
• Other types of data

Convolutional Neural Networks

Convolutional Neural Networks, or CNNs, were designed to map image data to an output variable.

They have proven so effective that they are the go-to method for any type of prediction problem involving image data as an input. Convolutional Neural Networks expect and preserve the spatial relationship between pixels by learning internal feature representations using small squares of input data. Feature are learned and used across the whole image, allowing for the objects in the images to be shifted or translated in the scene and still detectable by the network.

Types of problems CNNs solve:

• Image data
• Classification prediction problems
• Regression prediction problems
• More generally, CNNs work well with data that has a spatial relationship. A spatial relation specifies how some object is located in space in relation to some reference object.

The CNN input is traditionally 2D, a field or matrix, but can also be changed to be 1D, allowing it to develop an internal representation of a one-dimensional sequence.

This allows the CNN to be used more generally on other types of data that has a spatial relationship. For example, there is an order relationship between words in a document of text. There is an ordered relationship in the time steps of a time series.

Although not specifically developed for non-image data, CNNs achieve state-of-the-art results on problems such as document classification used in sentiment analysis and related problems.

Try CNNs On:

• Images
• Text data
• Time series data
• Sequence input data

Recurrent Neural Networks

Recurrent Neural Networks, or RNNs, were designed to work with sequence prediction problems. Sequence prediction is a problem that involves using historical sequence information to predict the next value or values in the sequence. The sequence may be symbols like letters in a sentence or real values like those in a time series of prices.

Some examples of sequence prediction problems include:

• Sequence to sequence: predict stock prices based on series of historical data
• Sequence to vector: words in a sentence to sentiment
• Vector to sequence: create captions from an image
• Encoder -> Decoder
  • Sequence -> vector -> sequence: machine translation

Recurrent neural networks were traditionally difficult to train. The Long Short-Term Memory, or LSTM, network is perhaps the most successful RNN because it overcomes the problems of training a recurrent network and in turn has been used on a wide range of applications.

RNNs in general and LSTMs in particular have received the most success when working with sequences of words and paragraphs, generally called natural language processing.

This includes both sequences of text and sequences of spoken language represented as a time series. They are also used as generative models that require a sequence output, not only with text, but on applications such as generating handwriting.

Types of problems RNNs Solve:

• Text data
• Speech data
• Classification prediction problems
• Regression prediction problems
• Generative models

Recurrent neural networks are not appropriate for tabular datasets as you would see in a CSV file or spreadsheet. They are also not appropriate for image data input.

Don’t Use RNNs For:

• Tabular data
• Image data

RNNs and LSTMs have been tested on time series forecasting problems, but the results have been poor, to say the least. Autoregression methods, even linear methods often perform much better. LSTMs are often outperformed by simple MLPs applied on the same data.