Here are some clarifications on A2 that came up from
questions some of you asked during office hours.
In Q2, when I say x* is the minimum, I mean minimizer.
Also note that the quadratic is defined with + in all its terms,
which is different from the notes, where there is a - in the b^Tx term.
In all cases, the SD method follows
xnew = xold + step (-gradf), where step is positive
or
xnew = xold - step gradf
If the residual is defined as r = -Ax - b, then r = - gradf
and the update is
xnew = xold + step r (as in the notes).
For consistency, define the error as e = x* - x.
(Aside: if you define the residual as r = Ax + b, then r = gradf,
and the update is xnew = xold - step r, and the error is e = x - x*.
There is nothing really wrong with that, but the set of relations
r = -Ax - b, r = - gradf, xnew = xold + step r, e = x* - x
is preferred.)
In Q3a, you must give the coefficients of the quadratic polynomial
corresponding to the quadratic and linear terms,
but can skip giving the coefficient of the constant term explicitly.
In Q3a, the term “minimizer” should be “minimizer or maximizer”.
In Q3b, with the no sufficient decrease, and the descent direction,
you will show it is minimizer (and not maximizer).