WOLFRAM

Wolfram Language & System Documentation Center
Wolfram Language Home Page »

"KMeans" (Machine Learning Method)

Details & Suboptions

  • "KMeans" is a classic, simple, centroid-based clustering method. "KMeans" works when clusters have similar sizes and are locally and isotropically distributed around their centroid. When clusters have very different sizes, are anisotropic, are intertwined, or when outliers are present, it is likely that "KMeans" will give poor results.
  • The following plots show the results of the "KMeans" method applied to toy datasets:
  • The "KMeans" method aims to find k centroids defining k clusters. Each data point is assigned to its nearest centroid. All points assigned to a given centroid are forming a cluster.
  • The procedure to find the best k centroids is iterative. The search starts by using random centroids and assigning each point to its nearest centroid:
  • Once all clusters are defined, the mean of each cluster becomes a new centroid:
  • This procedure is repeated until the clusters remain unchanged. This iterative procedure is sometimes called "hard EM" (hard Expectation Maximization).
  • The "KMeans" method is similar to the "GaussianMixture" with a spherical covariance (that is, all clusters are isotropic and have the same size).
  • Since the initial centroids are chosen randomly, results might differ upon evaluation.
  • The suboption "InitialCentroids" can be used to specify the initial centroids as a list of data points.
  • The following suboption can be given:
  • "InitialCentroids" Automatica list of initial centroids

Examples

open allclose all

Basic Examples  (3)Summary of the most common use cases

Find exactly four clusters of nearby values using the "KMeans" clustering method:

In[1]:=1
✖
https://wolfram.com/xid/0dx1j16qpw-wxc5bs
Out[1]=1

Create random 2D vectors:

In[1]:=1
✖
https://wolfram.com/xid/0dx1j16qpw-q8s2rm
Out[1]=1

Plot computed clusters using the "KMeans" method:

In[2]:=2
✖
https://wolfram.com/xid/0dx1j16qpw-fj7crm
Out[2]=2

Train a ClassifierFunction on a list of strings:

In[1]:=1
✖
https://wolfram.com/xid/0dx1j16qpw-nog76a
Out[1]=1

Find the cluster assignments and gather the elements by their cluster:

In[2]:=2
✖
https://wolfram.com/xid/0dx1j16qpw-lkm67v
Out[2]=2
Out[2]=2

Options  (3)Common values & functionality for each option

DistanceFunction  (1)

Cluster data using Manhattan distance:

In[1]:=1
✖
https://wolfram.com/xid/0dx1j16qpw-xbfrhk
Out[1]=1

"InitialCentroids"  (2)

Generate a list of 100 random colors:

In[14]:=14
✖
https://wolfram.com/xid/0dx1j16qpw-p4t1fk
Out[14]=14

Cluster the colors without specifying the initial configuration of centroids using the "KMeans" method:

In[25]:=25
✖
https://wolfram.com/xid/0dx1j16qpw-f56t7r
Out[25]=25

Specify the initial colors to be used as centroids using the "KMeans" method:

In[27]:=27
✖
https://wolfram.com/xid/0dx1j16qpw-ng6mcd
Out[27]=27

Create random 2D vectors:

In[1]:=1
✖
https://wolfram.com/xid/0dx1j16qpw-h5fmvf
Out[1]=1

Find different clusterings of data using the "KMeans" method by varying the "InitialCentroids":

In[2]:=2
✖
https://wolfram.com/xid/0dx1j16qpw-7f42ki
Out[2]=2

Possible Issues  (1)Common pitfalls and unexpected behavior

Create and visualize noisy 2D moon-shaped training and test datasets:

In[1]:=1
✖
https://wolfram.com/xid/0dx1j16qpw-f8bvfg
Out[1]=1

Train a ClassifierFunction using "KMeans" for two clusters and find clusters in the test set:

In[2]:=2
✖
https://wolfram.com/xid/0dx1j16qpw-xpxr9m
Out[2]=2

Visualizing clusters indicates that "KMeans" performs poorly on intertwined clusters:

In[3]:=3
✖
https://wolfram.com/xid/0dx1j16qpw-ultqo9
Out[3]=3