# lasagne.layers.dnn¶

This module houses layers that require cuDNN to work. Its layers are not automatically imported into the lasagne.layers namespace: To use these layers, you need to import lasagne.layers.dnn explicitly.

Note that these layers are not required to use cuDNN: If cuDNN is available, Theano will use it for the default convolution and pooling layers anyway. However, they allow you to enforce the usage of cuDNN or use features not available in lasagne.layers.

class lasagne.layers.dnn.Pool2DDNNLayer(incoming, pool_size, stride=None, pad=(0, 0), ignore_border=True, mode='max', **kwargs)[source]

2D pooling layer

Performs 2D mean- or max-pooling over the two trailing axes of a 4D input tensor. This is an alternative implementation which uses theano.sandbox.cuda.dnn.dnn_pool directly.

Parameters: incoming : a Layer instance or tuple The layer feeding into this layer, or the expected input shape. pool_size : integer or iterable The length of the pooling region in each dimension. If an integer, it is promoted to a square pooling region. If an iterable, it should have two elements. stride : integer, iterable or None The strides between sucessive pooling regions in each dimension. If None then stride = pool_size. pad : integer or iterable Number of elements to be added on each side of the input in each dimension. Each value must be less than the corresponding stride. ignore_border : bool (default: True) This implementation never includes partial pooling regions, so this argument must always be set to True. It exists only to make sure the interface is compatible with lasagne.layers.MaxPool2DLayer. mode : string Pooling mode, one of ‘max’, ‘average_inc_pad’ or ‘average_exc_pad’. Defaults to ‘max’. **kwargs Any additional keyword arguments are passed to the Layer superclass.

Notes

The value used to pad the input is chosen to be less than the minimum of the input, so that the output of each pooling region always corresponds to some element in the unpadded input region.

This is a drop-in replacement for lasagne.layers.MaxPool2DLayer. Its interface is the same, except it does not support the ignore_border argument.

class lasagne.layers.dnn.MaxPool2DDNNLayer(incoming, pool_size, stride=None, pad=(0, 0), ignore_border=True, **kwargs)[source]

2D max-pooling layer

Subclass of Pool2DDNNLayer fixing mode='max', provided for compatibility to other MaxPool2DLayer classes.

class lasagne.layers.dnn.Pool3DDNNLayer(incoming, pool_size, stride=None, pad=(0, 0, 0), ignore_border=True, mode='max', **kwargs)[source]

3D pooling layer

Performs 3D mean- or max-pooling over the 3 trailing axes of a 5D input tensor. This is an alternative implementation which uses theano.sandbox.cuda.dnn.dnn_pool directly.

Parameters: incoming : a Layer instance or tuple The layer feeding into this layer, or the expected input shape. pool_size : integer or iterable The length of the pooling region in each dimension. If an integer, it is promoted to a square pooling region. If an iterable, it should have two elements. stride : integer, iterable or None The strides between sucessive pooling regions in each dimension. If None then stride = pool_size. pad : integer or iterable Number of elements to be added on each side of the input in each dimension. Each value must be less than the corresponding stride. ignore_border : bool (default: True) This implementation never includes partial pooling regions, so this argument must always be set to True. It exists only to make sure the interface is compatible with lasagne.layers.MaxPool2DLayer. mode : string Pooling mode, one of ‘max’, ‘average_inc_pad’ or ‘average_exc_pad’. Defaults to ‘max’. **kwargs Any additional keyword arguments are passed to the Layer superclass.

Notes

The value used to pad the input is chosen to be less than the minimum of the input, so that the output of each pooling region always corresponds to some element in the unpadded input region.

class lasagne.layers.dnn.MaxPool3DDNNLayer(incoming, pool_size, stride=None, pad=(0, 0, 0), ignore_border=True, **kwargs)[source]

3D max-pooling layer

Subclass of Pool3DDNNLayer fixing mode='max', provided for consistency to MaxPool2DLayer classes.

class lasagne.layers.dnn.Conv2DDNNLayer(incoming, num_filters, filter_size, stride=(1, 1), pad=0, untie_biases=False, W=lasagne.init.GlorotUniform(), b=lasagne.init.Constant(0.), nonlinearity=lasagne.nonlinearities.rectify, flip_filters=False, **kwargs)[source]

2D convolutional layer

Performs a 2D convolution on its input and optionally adds a bias and applies an elementwise nonlinearity. This is an alternative implementation which uses theano.sandbox.cuda.dnn.dnn_conv directly.

class lasagne.layers.dnn.Conv3DDNNLayer(incoming, num_filters, filter_size, stride=(1, 1, 1), pad=0, untie_biases=False, W=lasagne.init.GlorotUniform(), b=lasagne.init.Constant(0.), nonlinearity=lasagne.nonlinearities.rectify, flip_filters=False, **kwargs)[source]

3D convolutional layer

Performs a 3D convolution on its input and optionally adds a bias and applies an elementwise nonlinearity. This implementation uses theano.sandbox.cuda.dnn.dnn_conv3d directly.

class lasagne.layers.dnn.SpatialPyramidPoolingDNNLayer(incoming, pool_dims=[4, 2, 1], mode='max', **kwargs)[source]

Spatial Pyramid Pooling Layer

Performs spatial pyramid pooling (SPP) over the input. It will turn a 2D input of arbitrary size into an output of fixed dimension. Hence, the convolutional part of a DNN can be connected to a dense part with a fixed number of nodes even if the dimensions of the input image are unknown.

The pooling is performed over $$l$$ pooling levels. Each pooling level $$i$$ will create $$M_i$$ output features. $$M_i$$ is given by $$n_i * n_i$$, with $$n_i$$ as the number of pooling operation per dimension in level $$i$$, and we use a list of the $$n_i$$‘s as a parameter for SPP-Layer. The length of this list is the level of the spatial pyramid.

Parameters: incoming : a Layer instance or tuple The layer feeding into this layer, or the expected input shape. pool_dims : list of integers The list of $$n_i$$‘s that define the output dimension of each pooling level $$i$$. The length of pool_dims is the level of the spatial pyramid. mode : string Pooling mode, one of ‘max’, ‘average_inc_pad’ or ‘average_exc_pad’. Defaults to ‘max’. **kwargs Any additional keyword arguments are passed to the Layer superclass.

Notes

This layer should be inserted between the convolutional part of a DNN and its dense part. Convolutions can be used for arbitrary input dimensions, but the size of their output will depend on their input dimensions. Connecting the output of the convolutional to the dense part then usually demands us to fix the dimensions of the network’s InputLayer. The spatial pyramid pooling layer, however, allows us to leave the network input dimensions arbitrary. The advantage over a global pooling layer is the added robustness against object deformations due to the pooling on different scales.

References

 [1] He, Kaiming et al (2015): Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition. http://arxiv.org/pdf/1406.4729.pdf.
class lasagne.layers.dnn.BatchNormDNNLayer(incoming, axes='auto', epsilon=1e-4, alpha=0.1, beta=lasagne.init.Constant(0), gamma=lasagne.init.Constant(1), mean=lasagne.init.Constant(0), inv_std=lasagne.init.Constant(1), **kwargs)[source]

Batch Normalization

This layer implements batch normalization of its inputs:

$y = \frac{x - \mu}{\sqrt{\sigma^2 + \epsilon}} \gamma + \beta$

This is a drop-in replacement for lasagne.layers.BatchNormLayer that uses cuDNN for improved performance and reduced memory usage.

Parameters: incoming : a Layer instance or a tuple The layer feeding into this layer, or the expected input shape axes : ‘auto’, int or tuple of int The axis or axes to normalize over. If 'auto' (the default), normalize over all axes except for the second: this will normalize over the minibatch dimension for dense layers, and additionally over all spatial dimensions for convolutional layers. Only supports 'auto' and the equivalent axes list, or 0 and (0,) to normalize over the minibatch dimension only. epsilon : scalar Small constant $$\epsilon$$ added to the variance before taking the square root and dividing by it, to avoid numerical problems. Must not be smaller than 1e-5. alpha : scalar Coefficient for the exponential moving average of batch-wise means and standard deviations computed during training; the closer to one, the more it will depend on the last batches seen beta : Theano shared variable, expression, numpy array, callable or None Initial value, expression or initializer for $$\beta$$. Must match the incoming shape, skipping all axes in axes. Set to None to fix it to 0.0 instead of learning it. See lasagne.utils.create_param() for more information. gamma : Theano shared variable, expression, numpy array, callable or None Initial value, expression or initializer for $$\gamma$$. Must match the incoming shape, skipping all axes in axes. Set to None to fix it to 1.0 instead of learning it. See lasagne.utils.create_param() for more information. mean : Theano shared variable, expression, numpy array, or callable Initial value, expression or initializer for $$\mu$$. Must match the incoming shape, skipping all axes in axes. See lasagne.utils.create_param() for more information. inv_std : Theano shared variable, expression, numpy array, or callable Initial value, expression or initializer for $$1 / \sqrt{ \sigma^2 + \epsilon}$$. Must match the incoming shape, skipping all axes in axes. See lasagne.utils.create_param() for more information. **kwargs Any additional keyword arguments are passed to the Layer superclass.

batch_norm_dnn
Convenience function to apply batch normalization

Notes

This layer should be inserted between a linear transformation (such as a DenseLayer, or Conv2DLayer) and its nonlinearity. The convenience function batch_norm_dnn() modifies an existing layer to insert cuDNN batch normalization in front of its nonlinearity.

For further information, see lasagne.layers.BatchNormLayer. This implementation is fully compatible, except for restrictions on the axes and epsilon arguments.

lasagne.layers.dnn.batch_norm_dnn(layer, **kwargs)[source]

Apply cuDNN batch normalization to an existing layer. This is a drop-in replacement for lasagne.layers.batch_norm(); see there for further information.

Parameters: layer : A Layer instance The layer to apply the normalization to; note that it will be modified as specified in lasagne.layers.batch_norm() **kwargs Any additional keyword arguments are passed on to the BatchNormDNNLayer constructor. BatchNormDNNLayer or NonlinearityLayer instance A batch normalization layer stacked on the given modified layer, or a nonlinearity layer stacked on top of both if layer was nonlinear.