WOLFRAM

Wolfram Language & System Documentation Center

"GradientBoostedTrees" (Machine Learning Method)

  • Method for Classify and Predict.
  • Predict the value or class of an example using an ensemble of decision trees.
  • Trees are trained sequentially following the boosting meta-algorithm.

Details & Suboptions

  • Gradient boosting is a machine learning technique for regression and classification problems that produces a prediction model in the form of an ensemble of trees. Trees are trained sequentially with the goal of compensating the weaknesses of previous trees. The current implementation uses the LightGBM framework in the back end.
  • The following options can be given:
  • MaxTrainingRounds 50number of boosting rounds
    "BoostingMethod" "Gradient"the method to use
    "L1Regularization"0L1 regularization parameter
    "L2Regularization"0L2 regularization parameter
    "LeafSize" Automaticminimum number of data samples in one leaf
    "LearningRate"Automaticlearning rate used in gradient descent
    "LeavesNumber" Automaticminimum number of leaves in one tree
    "MaxDepth" 6maximum depth of each tree
  • Possible settings for "BoostingMethod" include "Gradient", "GradientOneSideSampling", and "DART" (i.e. Dropouts meet Multiple Additive Regression Trees).

Examples

open all close all

Basic Examples  (2)

Train a predictor function on labeled examples:

Obtain information about the predictor:

Predict a new example:

Generate some data and visualize it:

Train a predictor function on it:

Compare the data with the predicted values and look at the standard deviation:

Options  (8)

"BoostingMethod"  (1)

Train two classifiers on the "WineQuality" training set using a different "BoostingMethod" for each, and compare the training time:

Compare the accuracy on a test set:

"LeafSize"  (2)

Train a predictor function using the "LeafSize" option:

Train two classifiers on the "Titanic" dataset by changing the value of "LeafSize":

Look at how the performance increases:

"LeavesNumber"  (1)

Generate a labeled training set:

Train two predictors using a different "LeavesNumber" for each:

Compare the performance:

"MaxDepth"  (2)

Use the "MaxDepth" option to train a classifier:

Use the "BostonHomes" training set to train two predictors with a different "MaxDepth" for each:

Compare the "ComparisonPlot" on a test set:

MaxTrainingRounds  (2)

Use the MaxTrainingRounds option to train a classifier:

Train two classifiers on the "Mushroom" dataset by changing the value of MaxTrainingRounds:

Look at how the performance increases: