WOLFRAM

Wolfram Language & System Documentation Center
Wolfram Language Home Page »

"SupportVectorMachine" (Machine Learning Method)

  • Method for Classify.
  • Models class probabilities by finding a hyperplane that separates the training data into two classes using a maximum-margin hyperplane.

Details & Suboptions

  • Support vector machines are binary classifiers. A kernel function is used to extract features from the examples. At training time, the method finds the maximum-margin hyperplane that separates classes. The multiclass classification problem is reduced to a set of binary classification problems (using a one-vs.-one or a one-vs.-all strategy). The current implementation uses the LibSVM framework in the back end.
  • The option "KernelType" allows you to choose the type of kernel to use. Possible settings for "KernelType" include:
  • "RadialBasisFunction"uses an exponential radial basis function as kernel
    "Polynomial"uses a polynomial function as kernel
    "Sigmoid"uses a sigmoidal function as kernel
    "Linear"uses a linear function as kernel
  • The kernel "RadialBasisFunction" takes the form:
  • The kernel "Polynomial" takes the form:
  • The kernel "Sigmoid" takes the form:
  • The kernel "Linear" takes the form:
  • The following options can be given:
  • "BiasParameter"1bias term c in polynomial and sigmoid kernels
    "GammaScalingParameter" Automaticthe parameter in the preceding kernels
    "KernelType" "RadialBasisFunction"the kernel to use to map to higher dimensions
    "MulticlassStrategy" Automaticthe strategy to use to obtain a multiclass classifier
    "PolynomialDegree" 3the degree of the polynomial d in the polynomial kernel
  • Possible settings for "MulticlassStrategy" include:
  • "OneVersusOne"train a binary classifier for each pair of classes
    "OneVersusAll"train one binary classifier for each class
  • The "GammaScalingParameter" controls the influence of the support vectors. Large values of gamma mean small radius of influence.
  • The "PolynomialDegree" option is specific to the polynomial kernel type.
  • The "MulticlassStrategy" option is used to generalize binary classifiers to a multiclass ones. The "OneVersusOne" strategy tests each class again each other, while the "OneVersusAll" strategy only test each class against the rest of the training set.

Examples

open allclose all

Basic Examples  (2)Summary of the most common use cases

Train a classifier function on labeled examples:

In[1]:=1
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-wxc5bs
Out[1]=1

Obtain information about the classifier:

In[2]:=2
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-3656v8
Out[2]=2

Classify a new example:

In[3]:=3
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-tgze8s
Out[3]=3

Generate some data that is not linearly separable:

In[1]:=1
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-qxqbpt
In[2]:=2
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-f99uln

Visualize it:

In[3]:=3
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-fcsxx1
Out[3]=3

Train a classifier on this dataset:

In[4]:=4
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-qv36d2
Out[4]=4

Plot the training set and the probability distribution of each class as a function of the features:

In[5]:=5
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-ebncen
Out[5]=5

Options  (5)Common values & functionality for each option

"GammaScalingParameter"  (1)

Train a classifier with a specific value for the "GammaScalingParameter" suboption:

In[1]:=1
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-32di5h
Out[1]=1

The "GammaScalingParameter" controls the influence of the support vectors.

Generate some data and visualize it:

In[2]:=2
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-k6bh0g
In[3]:=3
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-ognd2t
Out[3]=3

Train two classifiers by changing the "GammaScalingParameter":

In[4]:=4
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-iiulex
Out[4]=4
In[5]:=5
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-y7vhhq
Out[5]=5

Look at how they perform on a test set to see how the radius of influence has changed:

In[6]:=6
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-ybrfh4
In[7]:=7
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-7oxbk4
Out[7]=7

"KernelType"  (2)

Train a classifier using a specific "KernelType":

In[1]:=1
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-773w9i
Out[1]=1

Train two classifiers using different instances of "KernelType":

In[1]:=1
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-v5oels
In[2]:=2
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-83pxw8
In[3]:=3
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-ggonvm

Compare their performance:

In[4]:=4
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-hdclu
Out[4]=4
Out[4]=4

"MulticlassStrategy"  (1)

Use the "FisherIris" dataset to train two classifiers with different "MulticlassStrategy" options:

In[1]:=1
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-wrlkvg
In[2]:=2
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-xzyrpy

Look at their accuracy on a test set. The "OneVersusOne" option typically performs better.

In[3]:=3
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-l2rzfh
In[4]:=4
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-coq7kb
Out[4]=4

"PolynomialDegree"  (1)

Use the "Mushroom" training set to train two classifiers using different degrees for the polynomial kernel type:

In[1]:=1
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-u80tc7
In[2]:=2
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-iswh3m

Compare the corresponding training times:

In[3]:=3
✖
https://wolfram.com/xid/0ecilgt0d1grxo0-zxg7vy
Out[3]=3