Multi-layer Perceptron

A multi-layer perceptron is a simple fully-connected neural network where each of the neurons from a given layer is connected to each of the neurons of the next layer. While this represents a very basic type of architecture, it can become computationally expensive very quickly with increasing number of layers and neurons per layers.

PyTorch module

When using PyTorch, it is customary to build the model within a class. This class described above should be initialized prior from starting the actual training.

class hyppo.dnnmodels.pytorch.mlp. PytorchMLP ( data = None , prms = None , ** kwargs ) [source]

PyTorch class for multi-layer perceptron model.

Examples

Let’s consider the following random hyperparameters:

>>> prms = {'layers':2,'nodes':[20,20],'activation':[torch.nn.ReLU,torch.nn.ReLU],'dropout':[0.1,0.1]}

Let’s consider the input dataset to be the CIFAR10 dataset, we can load the PyTorch dataloader as follows:

>>> from hyppo.datasets.cifar10 import get_data
>>> from hyppo.datasets.loaders import get_loader
>>> data = get_data(library='pt')
>>> loaders = get_loader(data, batch=10)

The MLP model can then be built as follows (remember, the CIFAR10 dataset requires the n_classes parameter to be set to 10):

>>> from hyppo.dnnmodels.pytorch.mlp import PytorchMLP
>>> PytorchMLP(data['train'],prms,n_classes=10)
PytorchMLP(
  (layers): Sequential(
    (0): Linear(in_features=1024, out_features=20, bias=True)
    (1): ReLU()
    (2): Dropout(p=0.1, inplace=False)
    (3): Linear(in_features=20, out_features=20, bias=True)
    (4): ReLU()
    (5): Dropout(p=0.1, inplace=False)
    /
    (6): Linear(in_features=20, out_features=10, bias=True)
  )
)

Methods Summary

__init__ ([data, prms])

Initialize the model based on data properties and hyperparameter set.

forward (data)

Function that performs the forward propagation.

Methods Documentation

__init__ ( data = None , prms = None , ** kwargs ) [source]

Initialize the model based on data properties and hyperparameter set.

Parameters :
data DataLoader

Training data.

prms dict

Input set of hyperparameter values.

n_classes int

If not None, number of output classes from the network.

forward ( data ) [source]

Function that performs the forward propagation. The input data are reshaped into a vector form then fed to the network. The output vector is then reshaped according to the output format.

Parameters :
data Tensor

Input data batch.

Returns :
out Tensor

Output data out of the neural network.

Tensorflow module