fminBFGS

fminBFGS(func, X0, [fprime], [gtol=1e-5], [norm], [epsilon], [maxIter], [xrtol=0], [c1=1e-4], [c2=0.9])

func is the function to minimize. The return value of the function must be numeric type.

X0 is a numeric scalar or vector indicating the initial guess.

fprime (optional) is the gradient of func. If not provided, then func returns the function value and the gradient.

gtol (optional) is a postive number. Iteration will terminates if gradient norm is less than gtol. The default value is 1e-5.

norm (optional) is a positive number indicating the order of norm. Maximum norm is used by default.

epsilon (optional) is a positive number indicating the step size used for numerically calculating the gradient. The default value is 1.4901161193847656e-08.

maxIter (optional) is a non-negative integer indicating the maximum number of iterations. The default value is X0 * 200.

xrtol (optional) is a non-negative number indicating the relative tolerance. Iteration will terminate if step size is less than xk * xrtol where xk is the current parameter vector. The default value is 0.

c1 (optional) is a number in (0,1) indicating the parameter for Armijo condition rule. The default value is 1e-4.

c2 (optional) is a number in (0,1) indicating the parameter for curvature condition rule. The default value is 0.9. Note that c2 must be greater than c1.

Minimize a function using the BFGS algorithm.

Return value: A dictionary with the following members:

• xopt: A floating-point vector indicating the parameters of the minimum.

• fopt: A floating-point scalar indicating the value of func at the minimum, i.e., fopt=func(xopt).

• gopt: A floating-point vector indicating the gradient at the minimum. gopt=func'(xopt), which should be near 0.

• Hinv: A floating-point matrix representing the inverse Hessian matrix.

• iterations: Number of iterations.

• fcalls: Number of function calls made.

• gcalls: Number of gradient calls made.

• warnFlag: An integer, which can be

  • 0: Minimization performed.

  • 1: Maximum number of iterations exceeded.

  • 2: Line search failed or extreme values encountered.

  • 3: NULL result encountered.

Minimize function quadratic_cost using the BFGS algorithm:

def quadratic_cost(x, Q) {
	return dot(dot(x, Q), x)
}

def quadratic_cost_grad(x, Q) {
	return 2 * dot(Q, x)
}

x0 = [-3, -4]
cost_weight = diag([1., 10.])

fminBFGS(quadratic_cost{,cost_weight}, x0, quadratic_cost_grad{,cost_weight})

Output:

fcalls->8
warnFlag->0
xopt->[0.000002859166,-4.54371E-7]
Hinv->
#0              #1             
0.508225788096  -0.001307222772
-0.001307222772 0.050207740748 

gopt->[0.000005718332,-0.000009087439]
fopt->1.0E-11
gcalls->8
iterations->7