lasagne.layers.cuda_convnet

This module houses layers that require pylearn2 <https://deeplearning.net/software/pylearn2> to work. Its layers are not automatically imported into the lasagne.layers namespace: To use these layers, you need to import lasagne.layers.cuda_convnet explicitly.

class lasagne.layers.cuda_convnet.Conv2DCCLayer(incoming, num_filters, filter_size, stride=(1, 1), pad=0, untie_biases=False, W=None, b=lasagne.init.Constant(0.), nonlinearity=lasagne.nonlinearities.rectify, dimshuffle=True, flip_filters=False, partial_sum=1, **kwargs)[source]

2D convolutional layer

Performs a 2D convolution on its input and optionally adds a bias and applies an elementwise nonlinearity. This is an alternative implementation which uses the cuda-convnet wrappers from pylearn2: pylearn2.sandbox.cuda_convnet.filter_acts.FilterActs.

Parameters:

incoming : a Layer instance or a tuple

The layer feeding into this layer, or the expected input shape. This layer expects a 4D tensor as its input, with shape (batch_size, num_input_channels, input_rows, input_columns). If automatic dimshuffling is disabled (see notes), the shape should be (num_input_channels, input_rows, input_columns, batch_size) instead (c01b axis order).

num_filters : int

The number of learnable convolutional filters this layer has.

filter_size : int or iterable of int

An integer or a 2-element tuple specifying the size of the filters. This layer does not support non-square filters.

stride : int or iterable of int

An integer or a 2-element tuple specifying the stride of the convolution operation. This layer does not support using different strides along both axes.

pad : int, iterable of int, ‘full’, ‘same’ or ‘valid’ (default: 0)

By default, the convolution is only computed where the input and the filter fully overlap (a valid convolution). When stride=1, this yields an output that is smaller than the input by filter_size - 1. The pad argument allows you to implicitly pad the input with zeros, extending the output size.

A single integer results in symmetric zero-padding of the given size on all borders. This layer does not support using different amounts of padding along both axes, but for compatibility to other layers you can still specify the padding as a tuple of two same-valued integers.

'full' pads with one less than the filter size on both sides. This is equivalent to computing the convolution wherever the input and the filter overlap by at least one position.

'same' pads with half the filter size (rounded down) on both sides. When stride=1 this results in an output size equal to the input size. Even filter size is not supported.

'valid' is an alias for 0 (no padding / a valid convolution).

Note that 'full' and 'same' can be faster than equivalent integer values due to optimizations by Theano.

untie_biases : bool (default: False)

If False, the layer will have a bias parameter for each channel, which is shared across all positions in this channel. As a result, the b attribute will be a vector (1D).

If True, the layer will have separate bias parameters for each position in each channel. As a result, the b attribute will be a 3D tensor.

W : Theano shared variable, expression, numpy array or callable

Initial value, expression or initializer for the weights. These should be a 4D tensor with shape (num_filters, num_input_channels, filter_rows, filter_columns). If automatic dimshuffling is disabled (see notes), the shape should be (num_input_channels, input_rows, input_columns, num_filters) instead (c01b axis order). See lasagne.utils.create_param() for more information.

b : Theano shared variable, expression, numpy array, callable or None

Initial value, expression or initializer for the biases. If set to None, the layer will have no biases. Otherwise, biases should be a 1D array with shape (num_filters,) if untied_biases is set to False. If it is set to True, its shape should be (num_filters, output_rows, output_columns) instead. See lasagne.utils.create_param() for more information.

nonlinearity : callable or None

The nonlinearity that is applied to the layer activations. If None is provided, the layer will be linear.

dimshuffle : bool (default: True)

If True, the layer will automatically apply the necessary dimshuffle operations to deal with the fact that the cuda-convnet implementation uses c01b (batch-size-last) axis order instead of bc01 (batch-size-first), which is the Lasagne/Theano default. This makes the layer interoperable with other Lasagne layers.

If False, this automatic dimshuffling is disabled and the layer will expect its input and parameters to have c01b axis order. It is up to the user to ensure this. ShuffleBC01ToC01BLayer and ShuffleC01BToBC01Layer can be used to convert between bc01 and c01b axis order.

flip_filters : bool (default: False)

Whether to flip the filters and perform a convolution, or not to flip them and perform a correlation. Flipping adds a bit of overhead, so it is disabled by default. In most cases this does not make a difference anyway because the filters are learnt. However, flip_filters should be set to True if weights are loaded into it that were learnt using a regular lasagne.layers.Conv2DLayer, for example.

partial_sum : int or None (default: 1)

This value tunes the trade-off between memory usage and performance. You can specify any positive integer that is a divisor of the output feature map size (i.e. output rows times output columns). Higher values decrease memory usage, but also performance. Specifying 0 or None means the highest possible value will be used. The Lasagne default of 1 gives the best performance, but also the highest memory usage.

More information about this parameter can be found in the cuda-convnet documentation.

**kwargs

Any additional keyword arguments are passed to the Layer superclass.

Notes

The cuda-convnet convolution implementation has several limitations:

  • only square filters are supported.
  • only identical strides in the horizontal and vertical direction are supported.
  • the number of filters must be a multiple of 16.
  • the number of input channels must be even, or less than or equal to 3.
  • if the gradient w.r.t. the input is to be computed, the number of channels must be divisible by 4.
  • performance is optimal when the batch size is a multiple of 128 (but other batch sizes are supported).
  • this layer only works on the GPU.

The cuda-convnet convolution implementation uses c01b (batch-size-last) axis order by default. The Theano/Lasagne default is bc01 (batch-size-first). This layer automatically adds the necessary dimshuffle operations for the input and the parameters so that it is interoperable with other layers that assume bc01 axis order. However, these additional dimshuffle operations may sometimes negatively affect performance. For this reason, it is possible to disable them by setting dimshuffle=False. In this case, the user is expected to manually ensure that the input and parameters have the correct axis order. ShuffleBC01ToC01BLayer and ShuffleC01BToBC01Layer can be used to convert between bc01 and c01b axis order.

Attributes

W (Theano shared variable or expression) Variable or expression representing the filter weights.
b (Theano shared variable or expression) Variable or expression representing the biases.
class lasagne.layers.cuda_convnet.MaxPool2DCCLayer(incoming, pool_size, stride=None, ignore_border=False, dimshuffle=True, **kwargs)[source]

2D max-pooling layer

Performs 2D max-pooling over the two trailing axes of a 4D input tensor (or over axis 1 and 2 if dimshuffle=False, see notes). This is an alternative implementation which uses the cuda-convnet wrappers from pylearn2: pylearn2.sandbox.cuda_convnet.pool.MaxPool.

Parameters:

incoming : a Layer instance or tuple

The layer feeding into this layer, or the expected input shape.

pool_size : integer or iterable

The length of the pooling region in each dimension. If an integer, it is promoted to a square pooling region. If an iterable, it should have two elements. This layer does not support non-square pooling regions.

stride : integer, iterable or None

The strides between sucessive pooling regions in each dimension. If None then stride = pool_size. This layer does not support using different strides along both axes.

pad : integer or iterable (default: 0)

This implementation does not support custom padding, so this argument must always be set to 0. It exists only to make sure the interface is compatible with lasagne.layers.MaxPool2DLayer.

ignore_border : bool (default: False)

This implementation always includes partial pooling regions, so this argument must always be set to False. It exists only to make sure the interface is compatible with lasagne.layers.MaxPool2DLayer.

dimshuffle : bool (default: True)

If True, the layer will automatically apply the necessary dimshuffle operations to deal with the fact that the cuda-convnet implementation uses c01b (batch-size-last) axis order instead of bc01 (batch-size-first), which is the Lasagne/Theano default. This makes the layer interoperable with other Lasagne layers.

If False, this automatic dimshuffling is disabled and the layer will expect its input and parameters to have c01b axis order. It is up to the user to ensure this. ShuffleBC01ToC01BLayer and ShuffleC01BToBC01Layer can be used to convert between bc01 and c01b axis order.

**kwargs

Any additional keyword arguments are passed to the Layer superclass.

Notes

The cuda-convnet max-pooling implementation has several limitations:

  • only square pooling regions are supported.
  • only identical strides in the horizontal and vertical direction are supported.
  • only square inputs are supported. (This limitation does not exist for the convolution implementation.)
  • partial pooling regions are always included (ignore_border is forced to False).
  • custom padding is not supported (pad is forced to 0).
  • this layer only works on the GPU.

The cuda-convnet pooling implementation uses c01b (batch-size-last) axis order by default. The Theano/Lasagne default is bc01 (batch-size-first). This layer automatically adds the necessary dimshuffle operations for the input and the parameters so that it is interoperable with other layers that assume bc01 axis order. However, these additional dimshuffle operations may sometimes negatively affect performance. For this reason, it is possible to disable them by setting dimshuffle=False. In this case, the user is expected to manually ensure that the input and parameters have the correct axis order. ShuffleBC01ToC01BLayer and ShuffleC01BToBC01Layer can be used to convert between bc01 and c01b axis order.

class lasagne.layers.cuda_convnet.ShuffleBC01ToC01BLayer(incoming, name=None)[source]

shuffle 4D input from bc01 (batch-size-first) order to c01b (batch-size-last) order.

This layer can be used for interoperability between c01b and bc01 layers. For example, MaxPool2DCCLayer and Conv2DCCLayer operate in c01b mode when they are created with dimshuffle=False.

Parameters:

incoming : a Layer instance or tuple

The layer feeding into this layer, or the expected input shape.

**kwargs

Any additional keyword arguments are passed to the Layer superclass.

lasagne.layers.cuda_convnet.bc01_to_c01b[source]

alias of ShuffleBC01ToC01BLayer

class lasagne.layers.cuda_convnet.ShuffleC01BToBC01Layer(incoming, name=None)[source]

shuffle 4D input from c01b (batch-size-last) order to bc01 (batch-size-first) order.

This layer can be used for interoperability between c01b and bc01 layers. For example, MaxPool2DCCLayer and Conv2DCCLayer operate in c01b mode when they are created with dimshuffle=False.

Parameters:

incoming : a Layer instance or tuple

The layer feeding into this layer, or the expected input shape.

**kwargs

Any additional keyword arguments are passed to the Layer superclass.

lasagne.layers.cuda_convnet.c01b_to_bc01[source]

alias of ShuffleC01BToBC01Layer

class lasagne.layers.cuda_convnet.NINLayer_c01b(incoming, num_units, untie_biases=False, W=lasagne.init.GlorotUniform(), b=lasagne.init.Constant(0.), nonlinearity=lasagne.nonlinearities.rectify, **kwargs)[source]

Network-in-network layer with c01b axis ordering.

This is a c01b version of lasagne.layers.NINLayer.

Parameters:

incoming : a Layer instance or a tuple

The layer feeding into this layer, or the expected input shape

num_units : int

The number of units of the layer

untie_biases : bool

If False, the network has a single bias vector similar to a dense layer. If True, a separate bias vector is used for each spatial position.

W : Theano shared variable, numpy array or callable

An initializer for the weights of the layer. If a shared variable or a numpy array is provided the shape should be (num_units, num_input_channels). See lasagne.utils.create_param() for more information.

b : Theano shared variable, numpy array, callable or None

An initializer for the biases of the layer. If a shared variable or a numpy array is provided the correct shape is determined by the untie_biases setting. If untie_biases is False, then the shape should be (num_units,). If untie_biases is True then the shape should be (num_units, rows, columns). If None is provided the layer will have no biases. See lasagne.utils.create_param() for more information.

nonlinearity : callable or None

The nonlinearity that is applied to the layer activations. If None is provided, the layer will be linear.

**kwargs

Any additional keyword arguments are passed to the Layer superclass.