Create a CrossEntropyLossLayer object that takes a probability vector and an index:

Create a CrossEntropyLossLayer where the input is a probability vector and the target is an index:

Apply it to an input and a target:

Create a CrossEntropyLossLayer that operates on vectors generated from strings:

Apply it to an input and a target:

Completely correct predictions produce a loss of 0:

Create a CrossEntropyLossLayer where the input and target are single probabilities:

Apply it to an input and a target:

Thread the layer over a batch of inputs:

Create a CrossEntropyLossLayer where the input is a probability vector and the target is an index:

Apply it to an input and a target:

Create a CrossEntropyLossLayer where the input is a probability vector and the target is a probability vector:

Apply it to an input and a target:

Create a CrossEntropyLossLayer where the input and target are images representing matrices of binary class probabilities:

Apply the layer to an input and target:

Measure all possible losses from inputs and targets from a small set:

The input-target pairs with the smallest losses:

The input-target pairs with the largest losses:

Create a graph containing a CrossEntropyLossLayer in which the input is a 3-channel image in which each color channel represents a class, and the target is a matrix of indices representing the correct class for each pixel:

Measure the loss on a target image and matrix in which the areas that are predominantly red, green, and blue match the indices 1, 2, and 3, respectively, in the target:

Permuting the colors makes the per-pixel distributions disagree with the target matrix and increases the loss:

CrossEntropyLossLayer["Binary"] is used automatically by NetTrain when the final activation used for an output is an ElementwiseLayer[LogisticSigmoid]. Create a network that takes a pair of numbers and produces either True or False:

Train the network to decide if the first number in the pair is greater than the second:

The training network automatically constructed by NetTrain contains a binary-type CrossEntropyLossLayer:

Show the behavior of the trained network on the plane:

Plot the underlying probability learned by the network:

CrossEntropyLossLayer["Index"] is used automatically by NetTrain when the final activation used for an output is a SoftmaxLayer. Create an artificial dataset from three normally distributed clusters:

Plot the dataset:

The training data consists of rules mapping the point to the cluster it is in:

Create a net to compute the probability of a point lying in each cluster, using a "Class" decoder to classify the input as either Red, Green or Blue:

Train the net on the data:

The training network automatically constructed by NetTrain contains an index-type CrossEntropyLossLayer:

Evaluate the net on the centers of each cluster:

Show the contours in feature space in which each class reaches a posterior probability of 0.75:

Here is the function computed by CrossEntropyLossLayer["Binary"]:

Evaluate the function on some data:

This is equivalent to the following:

When the target is fixed at 1, the loss is minimized as the input approaches 1:

In general, the loss is minimized when the target approaches the input:

CrossEntropyLossLayer["Binary"] applied to scalar probabilities p is equivalent to CrossEntropyLossLayer["Probabilities"] applied to a vector probabilities {p,1-p}:

Demonstrate the same by substitution:

Here is the function computed by CrossEntropyLossLayer["Probabilities"]:

Evaluate the function on some data:

This is equivalent to the following:

CrossEntropyLossLayer["Probabilities"] is equivalent to CrossEntropyLossLayer["Index"] with a one-hot encoding:

Evaluating them on sparse label data gives the same result:

CrossEntropyLossLayer["Index"] is typically faster to evaluate than CrossEntropyLossLayer["Probabilities"] and can use significantly less memory when the number of classes is very large.