# lasagne.regularization¶

Functions to apply regularization to the weights in a network.

We provide functions to calculate the L1 and L2 penalty. Penalty functions take a tensor as input and calculate the penalty contribution from that tensor:

 l1 Computes the L1 norm of a tensor l2 Computes the squared L2 norm of a tensor

A helper function can be used to apply a penalty function to a tensor or a list of tensors:

 apply_penalty Computes the total cost for applying a specified penalty to a tensor or group of tensors.

Finally we provide two helper functions for applying a penalty function to the parameters in a layer or the parameters in a group of layers:

 regularize_layer_params_weighted Computes a regularization cost by applying a penalty to the parameters of a layer or group of layers, weighted by a coefficient for each layer. regularize_network_params Computes a regularization cost by applying a penalty to the parameters of all layers in a network.

## Examples¶

>>> import lasagne
>>> import theano.tensor as T
>>> import theano
>>> from lasagne.nonlinearities import softmax
>>> from lasagne.layers import InputLayer, DenseLayer, get_output
>>> from lasagne.regularization import regularize_layer_params_weighted, l2, l1
>>> from lasagne.regularization import regularize_layer_params
>>> layer_in = InputLayer((100, 20))
>>> layer1 = DenseLayer(layer_in, num_units=3)
>>> layer2 = DenseLayer(layer1, num_units=5, nonlinearity=softmax)
>>> x = T.matrix('x')  # shp: num_batch x num_features
>>> y = T.ivector('y') # shp: num_batch
>>> l_out = get_output(layer2, x)
>>> loss = T.mean(T.nnet.categorical_crossentropy(l_out, y))
>>> layers = {layer1: 0.1, layer2: 0.5}
>>> l2_penalty = regularize_layer_params_weighted(layers, l2)
>>> l1_penalty = regularize_layer_params(layer2, l1) * 1e-4
>>> loss = loss + l2_penalty + l1_penalty


## Helper functions¶

lasagne.regularization.apply_penalty(tensor_or_tensors, penalty, **kwargs)[source]

Computes the total cost for applying a specified penalty to a tensor or group of tensors.

Parameters: tensor_or_tensors : Theano tensor or list of tensors penalty : callable **kwargs keyword arguments passed to penalty. Theano scalar a scalar expression for the total penalty cost
lasagne.regularization.regularize_layer_params(layer, penalty, tags={'regularizable': True}, **kwargs)[source]

Computes a regularization cost by applying a penalty to the parameters of a layer or group of layers.

Parameters: layer : a Layer instances or list of layers. penalty : callable tags: dict Tag specifications which filter the parameters of the layer or layers. By default, only parameters with the regularizable tag are included. **kwargs keyword arguments passed to penalty. Theano scalar a scalar expression for the cost
lasagne.regularization.regularize_layer_params_weighted(layers, penalty, tags={'regularizable': True}, **kwargs)[source]

Computes a regularization cost by applying a penalty to the parameters of a layer or group of layers, weighted by a coefficient for each layer.

Parameters: layers : dict A mapping from Layer instances to coefficients. penalty : callable tags: dict Tag specifications which filter the parameters of the layer or layers. By default, only parameters with the regularizable tag are included. **kwargs keyword arguments passed to penalty. Theano scalar a scalar expression for the cost
lasagne.regularization.regularize_network_params(layer, penalty, tags={'regularizable': True}, **kwargs)[source]

Computes a regularization cost by applying a penalty to the parameters of all layers in a network.

Parameters: layer : a Layer instance. Parameters of this layer and all layers below it will be penalized. penalty : callable tags: dict Tag specifications which filter the parameters of the layer or layers. By default, only parameters with the regularizable tag are included. **kwargs keyword arguments passed to penalty. Theano scalar a scalar expression for the cost

## Penalty functions¶

lasagne.regularization.l1(x)[source]

Computes the L1 norm of a tensor

Parameters: x : Theano tensor Theano scalar l1 norm (sum of absolute values of elements)
lasagne.regularization.l2(x)[source]

Computes the squared L2 norm of a tensor

Parameters: x : Theano tensor Theano scalar squared l2 norm (sum of squared values of elements)