ENH AndersonAccelerator.accelerate() does not create new points most of the time about skglm

[QB3]

I guess this is linked to #33

[mathurinm]

That's where I noticed it, but it should be fixed in main first, before the other PR is merged, to make reviewing and benchmarking easier

@Badr-MOUFAD will have a look in the upcoming days

[QB3]

OK but just to be sure, this problem only appears in the group solver?

skglm/skglm/solvers/group_bcd_solver.py

Line 94 in 1f13430

w_acc, Xw_acc = accelerator.extrapolate(w, Xw)

For me it does not appear:

• in the single task solver, if the error is raised, then p_obj_acc is not computed

  skglm/skglm/solvers/cd_solver.py

  Line 301 in 1f13430

  except np.linalg.LinAlgError:

• in the multitask solver

  skglm/skglm/solvers/multitask_bcd_solver.py

  Line 266 in 1f13430

  try:

[Badr-MOUFAD]

AndersonAcceleration.extrapolate(w, Xw) returns the same (w, Xw) for iteration between 0 and K-1 (to construct the matrix of residuals) and return (w_acc, Xw_acc) at the K iteration.

Hence, we don't need to evaluate p_obj_acc (L96) for iteration 0 till K-1 since it is the same as p_obj.

skglm/skglm/solvers/group_bcd_solver.py

Lines 94 to 100 in f5e8154

	w_acc, Xw_acc = accelerator.extrapolate(w, Xw)
	p_obj = datafit.value(y, w, Xw) + penalty.value(w)
	p_obj_acc = datafit.value(y, w_acc, Xw_acc) + penalty.value(w_acc)
	if p_obj_acc < p_obj:
	w, Xw = w_acc, Xw_acc
	p_obj = p_obj_acc

OK but just to be sure, this problem only appears in the group solver?

Indeed @QB3, so far this problem is only related to the group solver as it is the only one using AndersonAccelaration.
However, AA class will be adopted in other solvers (#33). Hence the need to handle this problem.

---

thx @mathurinm, that's a nice catch!