alanthink / banditpylib Goto Github PK

Now CUCB can only take care of a LinearBandit. Can you help implement a CorrelatedBanditItf in bandits.utils file? Such that CUCB can deal with it and then let LinearBandit inherit from it.

Is contextual bandits in the scope of this library? Here is a paper for reference:

https://arxiv.org/pdf/1810.09558.pdf

In line 33 of https://github.com/Alanthink/banditpylib/blob/master/banditpylib/learners/bestarmid/fixconf/ordinarylearner/lilucb_heur.py, it uses

self.__delta = self._fail_prob/5

Why don't you let users input delta(confidence level) directly instead of fail_prob? Does this mean if I want to get confidence level 0.1 like used in the paper, I should specify fail_prob = 0.5?

I've read your paper titled Best Arm Identification in Linear Bandits with Linear Dimension Dependency and I'm interested in the implementation.
Adding the policy in the library would be great.

Hi,

Want to finish up Feraud's decentralized exploration algorithm before I take a break from implementing previous work & focus on something new.

Do you have tips on the correct way to implement a kind of meta learner (which references an arm selection scheme). In particular, the call to ArmSelection in line 13 refers to a selection scheme which returns eliminated actions. If it is okay for a learner to pull its own arms and observe the feedback, it should be fine, but I'm not sure if this is okay from a design perspective (from what I understand, learners primarily only observe stats from em_arm, and the protocol handles interface to the bandit & feedback?).

Alternatively, I could implement a separate protocol to take the place of this meta learner.

Here is the pseudocode for reference:

Would like to include a citation in something very preliminary, maybe some bibtex like SmPyBandit:

misc{SMPyBandits,
    title =   {{SMPyBandits: an Open-Source Research Framework for Single and Multi-Players Multi-Arms Bandits (MAB) Algorithms in Python}},
    author =  {Lilian Besson},
    year =    {2018},
    url =     {https://github.com/SMPyBandits/SMPyBandits/},
    howpublished = {Online at: \url{github.com/SMPyBandits/SMPyBandits}},
    note =    {Code at https://github.com/SMPyBandits/SMPyBandits/, documentation at https://smpybandits.github.io/}
}

Hi, thanks for this - it's a really awesome work! I'm interested in the decentralized/distributed bandit setup. Do you have any pointers for extending your code - e.g. to support communication between players?

Can you add MAB algorithms which can handle time-varying signals? Maybe non-stationary MAB algorithms are for this purpose?

I have uploaded a pylintrc file. I suggest run pylint first before submitting a pr. We should gradually change the existing files such that the coding style meets the one described in pylintrc.

Could be a long-term prospect if you are interested: http://www.jmlr.org/mloss/

Inspiration: https://perso.crans.org/besson/articles/SMPyBandits.pdf

Can you add a stopping rule for Thompson Sampling(ts.py)?