# UnconstrainedProblems Package

## Plotting Search Data

The utility functions FindMinimumPlot and FindRootPlot show search data for FindMinimum and FindRoot for one- and two-dimensional functions. They work with essentially the same arguments as FindMinimum and FindRoot except that they additionally take options, which affect the graphics functions they call to provide the plots, and they do not have the HoldAll attribute as do FindMinimum and FindRoot.

FindMinimumPlot[f,{x,x_{st}},opts] | plot the steps and the points at which the function f and any of its derivatives that were evaluated in FindMinimum[f,{x,x_{st}}], superimposed on a plot of f versus x; opts may include options from both FindMinimum and Plot |

FindMinimumPlot[f,{{x,x_{st}},{y,y_{st}}},opts] | plot the steps and the points at which the function f and any of its derivatives that were evaluated in FindMinimum[f,{{x,x_{st}},{y,y_{st}}}], superimposed on a contour plot of f as a function of x and y; opts may include options from both FindMinimum and ContourPlot |

FindRootPlot[f,{x,x_{st}},opts] | plot the steps and the points at which the function f and any of its derivatives that were evaluated in FindRoot[f,{x,x_{st}}], superimposed on a plot of f versus x; opts may include options from both FindRoot and Plot |

FindRootPlot[f,{{x,x_{st}},{y,y_{st}}},opts] | plot the steps and the points at which the function f and any of its derivatives that were evaluated in FindRoot[f,{{x,x_{st}},{y,y_{st}}}], superimposed on a contour plot of the merit function f as a function of x and y; opts may include options from both FindRoot and ContourPlot |

Note that to simplify processing and reduce possible confusion about the function f, FindRootPlot does not accept equations; it finds a root f=0.

Steps and evaluation points are color coded for easy detection as follows:

- Steps are shown with blue lines and blue points.
- Function evaluations are shown with green points.
- Gradient evaluations are shown with red points.
- Hessian evaluations are shown with cyan points.
- Residual function evaluations are shown with yellow points.
- Jacobian evaluations are shown with purple points.
- The search termination is shown with a large black point.

FindMinimumPlot and FindRootPlot uses the default argument "Plot" to return a plot of the steps.

FindMinimumPlot and FindRootPlot using the argument "SolutionData" returns an Association with keys {Plot,Solution,Statistics}, where:

- Plot is the graphics object shown.
- Solution is the result of FindMinimum or FindRoot.
- Statistics is a list of rules showing the number of steps and evaluations of the function and its derivatives.

## Test Problems

All the test problems presented in [MGH81] have been coded into the Wolfram Language in the Optimization`UnconstrainedProblems` package. A data structure is used so that the problems can be processed for solution and testing with FindMinimum and FindRoot in a seamless way. The lists of problems for FindMinimum and FindRoot are in $FindMinimumProblems and $FindRootProblems, respectively, and a problem can be accessed using GetFindMinimumProblem and GetFindRootProblem.

$FindMinimumProblems | list of problems that are appropriate for FindMinimum |

GetFindMinimumProblem[prob] | get the problem prob using the default size and starting values in a FindMinimumProblem data structure |

GetFindMinimumProblem[prob,{n,m}] | get the problem prob with n variables such that it is a sum of m squares in a FindMinimumProblem data structure |

GetFindMinimumProblem[prob,size,start] | |

get the problem prob with given size and starting value start in a FindMinimumProblem data structure | |

FindMinimumProblem[f,vars,opts,prob,size] | |

a data structure that contains a minimization problem to be solved by FindMinimum |

Accessing FindMinimum problems.

$FindRootProblems | list of problems that are appropriate for FindRoot |

GetFindRootProblem[prob] | get the problem prob using the default size and starting values in a FindRootProblem data structure |

GetFindRootProblem[prob,n] | get the problem prob with n variables (and n equations) in a FindRootProblem data structure |

GetFindRootProblem[prob,n,start] | get the problem prob with size n and starting value start in a FindRootProblem data structure |

FindRootProblem[f,vars,opts,prob,size] | |

a data structure that contains a minimization problem to be solved by FindRoot |

Accessing FindRoot problems.

GetFindMinimumProblem and GetFindRootProblem are both pass options to be used by other commands. They also accept the option Variables->vars which is used to specify what variables to use for the problems.

option name | default value | |

Variables | X_{#}& | a function that is applied to the integers to generate the variables for a problem with variables or a list of length containing the variables |

Once you have a FindMinimumProblem or FindRootProblem object, in addition to simply solving the problem, there are various tests that you can run.

ProblemSolve[p,opts] | solve the problem in p, giving the same output as FindMinimum or FindRoot |

ProblemStatistics[p,opts] | solve the problem, giving a list {sol,stats}, where sol is the output of ProblemSolve[p] and evals is a list of rules indicating the number of steps and evaluations used |

ProblemTime[p,opts] | solve the problem giving a list {sol,Time->time}, where sol is the output of ProblemSolve[p] and time is time taken to solve the problem; if time is less than a second, the problem will be solved multiple times to get an average timing |

ProblemTest[p,opts] | solve the problem, giving a list of rules including the step and evaluation statistics and time from ProblemStatistics[p] and ProblemTime[p] along with rules indicating the accuracy and precision of the solution as compared with a reference solution |

FindMinimumPlot[p,opts] | plot the steps and evaluation points for solving a FindMinimumProblem p |

FindRootPlot[p,opts] | plot the steps and evaluation points for solving a FindRootProblem p |

Operations with FindMinimumProblem and FindRootProblem data objects.

Any of the previous commands shown can take options that are passed on directly to FindMinimum or FindRoot and override any options for these functions which may have been specified when the problem was set up.

ProblemTest gives a way to easily compare two different methods for the same problem.

Most of the rules returned by these functions are self-explanatory, but a few require some description. Here is a table clarifying those rules.

"FunctionAccuracy" | the accuracy of the function value -Log[10,error in f] |

"FunctionPrecision" | the precision of the function value -Log[10,relative error in f] |

"SpatialAccuracy" | the accuracy in the position of the minimizer or root -Log[10,error in x] |

"SpatialPrecision" | the precision in the position of the minimizer or root -Log[10,relative error in x] |

"Messages" | a list of messages issued during the solution of the problem |

A very useful comparison is to see how a list of methods affect a particular problem. This is easy to do by setting up a FindMinimumProblem object and mapping a problem test over a list of methods.

It is possible to generate tables of how a particular method affects a variety of problems by mapping over the names in $FindMinimumProblems or $FindRootProblems.

The two cases where the spatial accuracy is shown as ERROR are for linear problems, which do not have an isolated minimizer. The one case, which has a spatial accuracy that is quite poor, has multiple minimizers, and the method goes to a different minimum than the reference one. Many of these functions have multiple local minima, so be aware that the error may be reported as large only because a method went to a different minimum than the reference one.