"KernelDensityEstimation" (Machine Learning Method)
- Method for LearnDistribution.
- Models probability density with a mixture of simple distributions.
Details & Suboptions
- "KernelDensityEstimation" is a nonparametric method that models the probability density of a numeric space with a mixture of simple distributions (called kernels) centered around each training example, as in KernelMixtureDistribution.
- The probability density function for a vector
is given by
for a kernel function
, kernel size
and a number of training examples m.
- The following options can be given:
-
Method "Fixed" kernel size method "KernelSize" Automatic size of the kernels when Method"Fixed" "KernelType" "Gaussian" type of kernel used "NeighborsNumber" Automatic kernel size expressed as a number of neighbors - Possible settings for "KernelType" include:
-
"Gaussian" each kernel is a Gaussian distribution "Ball" each kernel is a uniform distribution on a ball - Possible settings for Method include:
-
"Adaptive" kernel sizes can differ from each other "Fixed" all kernels have the same size - When "KernelType""Gaussian", each kernel is a spherical Gaussian (product of independent normal distributions
), and "KernelSize" h refers to the standard deviation of the normal distribution.
- When "KernelType""Ball", each kernel is a uniform distribution inside a sphere, and "KernelSize" refers to the radius of the sphere.
- The value of "NeighborsNumber"k is converted into kernel size(s), so that a kernel centered around a training example typically "contains" k other training examples. If "KernelType""Ball", "contains" refers to examples that are inside the ball. If "KernelType""Gaussian", "contains" refers to examples that are inside a ball of radius h
where n is the dimension of the data.
- When Method"Fixed" and "NeighborsNumber"k, a unique kernel size is found such that training examples contain on average k other examples.
- When Method"Adaptive" and "NeighborsNumber"k, each training example adapts its kernel size such that it contains about k other examples.
- Because of preprocessing, the "NeighborsNumber" option is typically a more convenient way to control kernel sizes than "KernelSize". When Method"Fixed", the value of "KernelSize" supersedes the value of "NeighborsNumber".
- Information[LearnedDistribution[…],"MethodOption"] can be used to extract the values of options chosen by the automation system.
- LearnDistribution[…,FeatureExtractor"Minimal"] can be used to remove most preprocessing and directly access the method.
Examples
open allclose allBasic Examples (3)Summary of the most common use cases
Train a "KernelDensityEstimation" distribution on a numeric dataset:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-6wc05d

Look at the distribution Information:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-zci6sv


https://wolfram.com/xid/0zyl7urlsg0ch3qjq-d4gaww

Obtain an option value directly:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-tm1l4s

Compute the probability density for a new example:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-37we6q

Plot the PDF along with the training data:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-jfmxrj

Generate and visualize new samples:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-trbtxb

Train a "KernelDensityEstimation" distribution on a two-dimensional dataset:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-g4novm

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-4ewt2p

Plot the PDF along with the training data:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-i4ttni

Use SynthesizeMissingValues to impute missing values using the learned distribution:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-og5slp


https://wolfram.com/xid/0zyl7urlsg0ch3qjq-feqqk1

Train a "KernelDensityEstimation" distribution on a nominal dataset:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-36gxxt

Because of the necessary preprocessing, the PDF computation is not exact:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-wbm3in


https://wolfram.com/xid/0zyl7urlsg0ch3qjq-jokluj

Use ComputeUncertainty to obtain the uncertainty on the result:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-29ml4l

Increase MaxIterations to improve the estimation precision:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-5lufi6

Options (4)Common values & functionality for each option
"KernelSize" (1)
Train a kernel mixture distribution with a kernel size of 0.2:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-vthdnl

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-g542sp

Evaluate the PDF of the distribution at a specific point:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-89877v

Visualize the PDF obtained after training a kernel mixture distribution with various kernel sizes:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-5bujoz

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-yv7v8r

"KernelType" (1)
Train a "KernelDensityEstimation" distribution with a "Ball" kernel:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-iu3rlx

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-sltk9u

Evaluate the PDF of the distribution at a specific point:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-v8vvn8

Visualize the PDF obtained after training a kernel mixture distribution with a "Ball" and a "Gaussian" kernel:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-hbjlx6

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-u1a9s8

Method (1)
Train a "KernelDensityEstimation" distribution with the "Adaptive" method:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-8n9io9

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-vfl8ow

Evaluate the PDF of the distribution at a specific point:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-di1qkj

Visualize the PDF obtained after training a kernel mixture distribution with a "Ball" and a "Gaussian" kernel:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-xakq2r

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-3grodf

"NeighborsNumber" (1)
Train a kernel mixture distribution with a kernel size of about 10 neighbors:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-eon6sf

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-bgqzxr

Evaluate the PDF of the distribution at a specific point:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-rhff7t

Visualize the PDF obtained after training a kernel mixture distribution with various kernel sizes expressed as neighbors numbers:

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-9hnbl1

https://wolfram.com/xid/0zyl7urlsg0ch3qjq-4fzufa
