Arms from previous batch keep appearing in new batches about ax

[esantorella]

A few hypotheses and suggestions:

• My best guess by far is that this is a result of using UCB, which will choose points with zero uncertainty if it believes they are sufficiently better than the other options. 10+/-0 has a higher upper confidence bound than 1+/-1. I would expect better performance and no repetition from leaving the acquisition function unspecified. Then Ax will default to using qLogNoisyExpectedImprovement, which shouldn't have this issue. (qLogNoisyExpectedImprovement is an improvement to qNoisyExpectedImprovement that has better numerical behavior.) If you really want to use UCB, although I don't recommend it, you could set a higher beta to prefer more uncertain arms.
• Are you using BoTorch < 0.9.3? 0.9.3 was released in November 2023. Before that, SingleTaskGP would not handle SEM data. I would have hoped Ax would raise an exception rather than silently ignoring the SEM data, but this is my next-best guess if specifying the SEM made no difference at all.
• Could you look at model.experiment.trials and confirm that all of the trials you expect to see are present and have status COMPLETED? From looking at your code, I don't think this is the problem, but I want to rule out that this is a similar issue to #2371 where trials haven't been properly attached.

[esantorella]

By the way, it looks like your use case could work well with the Service API, which is more user-friendly. Here is a tutorial.

[Samperius]

Hi, thanks for the advice.

• I tried with expected improvement and the issue does not reproduce with that acquisition function.
• My Botorch version: botorch 0.9.4
• All trials are seen as completed

[esantorella]

Hi, thanks for reaching out. Some amount of repetition is expected in a discrete search space like you have. When you pass in an SEM of None, Ax assumes that the data has some unknown amount of noise and tries to infer the amount of noise. Repeating an arm that performed well in the past provides informational value and helps the model infer the noise level. If your observations are noiseless, pass in an SEM of 0, and there shouldn't be repetition.

[Samperius]

Hi, thank you for the quick response. I tried to use the SEM of 0. The same behavior seems to reproduce.

initial arms: [Arm(name='0_0', parameters={'concentration': 2, 'temperature': 2, 'base': 'KOPiv', 'ligand': 'PPh3', 'solvent': 'p-Xylene'}), Arm(name='0_1', parameters={'concentration': 2, 'temperature': 2, 'base': 'KOAc', 'ligand': 'P(fur)3', 'solvent': 'BuOAc'}), Arm(name='0_2', parameters={'concentration': 1, 'temperature': 3, 'base': 'KOPiv', 'ligand': 'CgMe-PPh', 'solvent': 'DMAc'})] model is qUCB arm_name metric_name mean sem trial_index 0 0_0 yield 1.41 0.0 0 1 0_1 yield 9.87 0.0 0 2 0_2 yield 89.95 0.0 0

generating batch: [Arm(name='0_2', parameters={'concentration': 1, 'temperature': 3, 'base': 'KOPiv', 'ligand': 'CgMe-PPh', 'solvent': 'DMAc'}), Arm(name='1_0', parameters={'concentration': 3, 'temperature': 2, 'base': 'KOPiv', 'ligand': 'CgMe-PPh', 'solvent': 'DMAc'}), Arm(name='1_1', parameters={'concentration': 2, 'temperature': 2, 'base': 'CsOAc', 'ligand': 'CgMe-PPh', 'solvent': 'DMAc'})]

It seems that the candidates with the highest yield keep repeating. For instance, arm in the next trial:
0 1_0 yield 98.49 0.0 1
repeats still in trial 10 even though I always pass it to the same result (as I am using a pre-collected dataset and not actually perform the experiments again).

To me, it seems that there is still some uncertainty left in the GP of those repeating parameters resulting in UCB to expect high acquisition value remaining on those parameters.

To be noted that I get a numerical warning:
`.../python3.12/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:

A not p.d., added jitter of 1.0e-08 to the diagonal`