# LinearLayer

LinearLayer[n]

represents a trainable, fully connected net layer that computes with output vector of size n.

LinearLayer[{n1,n2,}]

represents a layer that outputs an array of dimensions n1×n2×.

leaves the dimensions of the output array to be inferred from context.

LinearLayer[n,opts]

includes options for initial weights and other parameters.

# Details and Options  • The following optional parameters can be included:
•  "Biases" Automatic initial vector of biases (b in w.x+b) "Weights" Automatic initial matrix of weights (w in w.x+b) LearningRateMultipliers Automatic learning rate multiplier(s) to apply to weights and/or biases
• When weights and biases are not explicitly specified or are given as Automatic, they are added automatically when NetInitialize or NetTrain is used.
• The setting "Biases"->None specifies that no biases should be used.
• If weights and biases have been added, LinearLayer[][input] explicitly computes the output from applying the layer.
• LinearLayer[][{input1,input2,}] explicitly computes outputs for each of the inputi.
• When given a NumericArray as input, the output will be a NumericArray.
• NetExtract can be used to extract weights and biases from a LinearLayer object.
• LinearLayer is typically used inside NetChain, NetGraph, etc.
• LinearLayer exposes the following ports for use in NetGraph etc.:
•  "Input" an array "Output" an array of size n1×n2×…
• LinearLayer[{}] specifies that the LinearLayer should produce a single real number.
• LinearLayer[n,"Input"->m] is the most common usage of LinearLayer and represents a LinearLayer that takes a vector of length m and produces a vector of length n.
• When it cannot be inferred from previous layers in a larger net, the option "Input"shape can be used to fix the input of LinearLayer. Possible forms for shape include:
•  "Real" a single real number m a vector of length m {m1,m2,…} an array of dimensions m1×m2×… NetEncoder[…] an encoder
• Options[LinearLayer] gives the list of default options to construct the layer. Options[LinearLayer[]] gives the list of default options to evaluate the layer on some data.
• Information[LinearLayer[]] gives a report about the layer.
• Information[LinearLayer[],prop] gives the value of the property prop of LinearLayer[]. Possible properties are the same as for NetGraph.

# Examples

open allclose all

## Basic Examples(2)

Create a LinearLayer whose output is a length-5 vector:

Create a randomly initialized LinearLayer:

Apply the layer to an input vector to produce an output vector:

## Scope(6)

Create a randomly initialized LinearLayer that takes length-3 vectors and produces 4×2 matrices:

Apply it to an input:

Create a randomly initialized LinearLayer that takes single real numbers and produces single real numbers:

Apply it to an input:

Create a randomly initialized LinearLayer that takes 3×2 matrices and produces single real numbers:

Apply it to an input:

Create a randomly initialized LinearLayer that takes a class label by using a NetEncoder:

Apply the layer to a member of the class:

The attached encoder produces 3-vectors in which one component is 1 and the others are 0:

Create a randomly initialized LinearLayer that takes an image by using a NetEncoder:

Use a specific weight matrix and bias vector:

Apply the layer to an input:

## Options(4)

### "Biases"(1)

Define and initialize a LinearLayer without biases:

This is equivalent to Dot:

### LearningRateMultipliers(2)

Create a LinearLayer with frozen weights and biases:

Train a net with this layer inside:

The weights and biases of the layer have been unchanged:

Create a LinearLayer with frozen weights, but free biases:

Train a net with this layer inside:

The weights have been unchanged, but the biases changed during training:

### "Weights"(1)

Specify a specific weight matrix, and use no biases:

Apply the layer to an input:

Extract the weight:

## Applications(1)

Create a two-layer perceptron by stacking linear layers in a NetChain:

Train the perceptron on the "MNIST" dataset of handwritten digits:

Classify unseen digits:

## Properties & Relations(2)

LinearLayer[n] can be specified as simply n in a NetChain:

For inputs and outputs that are vectors, LinearLayer computes:

Evaluate a LinearLayer on data:

Manually compute the same result:

## Possible Issues(3)

LinearLayer cannot be initialized until all its input and output dimensions are known: LinearLayer cannot accept symbolic inputs: A LinearLayer with output size n and input size m has a weight matrix of dimensions n×m:

If n and m are too large, there might not be enough system or GPU memory to initialize or train a net containing such a LinearLayer: 