Comments (5)
Thanks for the bug report!
I however tried different initializations and Python versions, but was unable to reproduce the error. Some questions:
- Are you using the latest version of this package? (that is: 1.7.2)
- Does it happen at the beginning (right after initialization) or after having called
fit
/partial_fit
multiple times? - Did you modify something in running the notebook?
- Are you running this in a regular computer/server/laptop, or is it some special architechture or cluster or something?
- Which joblib, NumPy and SciPy version are you using? In case they are old, have you tried updating them? (I tried NumPy 1.15 and SciPy 1.2)
It's weird that it would only happen with one class, as all classes in online
set those numbers through the same piece of code in utils
.
from contextualbandits.
Thanks for your time to check the problem.
I had figured it out, but I would still answer your questions before posting my solution.
- Are you using the latest version of this package? (that is: 1.7.2)
Yes, it is the latest version. - Does it happen at the beginning (right after initialization) or after having called fit/partial_fit multiple times?
I directly used the code in online_contextual_bandits.ipynb. It took place in
lst_actions[model] = simulate_rounds(models[model], lst_rewards[model], lst_actions[model], X, y, batch_st, batch_end)
- Did you modify something in running the notebook?
I didn't. - Are you running this in a regular computer/server/laptop, or is it some special architechture or cluster or something?
I ran it on a workstation, the arch should be x86_64. - Which joblib, NumPy and SciPy version are you using? In case they are old, have you tried updating them? (I tried NumPy 1.15 and SciPy 1.2)
I used joblib 0.12.5, numpy 1.15.4, scipy 1.1.0.
After debugging, it is caused by
def _calculate_beta_prior(nchoices):
return (3.0/nchoices, 4)
in contextualbandits/utils.py line 62.
In original version, it was 3/nchoices, when nchoices=159, it turns to 0. Now, I changed it to 3.0/nchoices=0.018. Then, it works like a charm.
Another thing I would like to report is that when active_choice='weighted' for AdaptiveGreedy, it would call AdaptiveExplorer._crit_active(). This would lead to another exception
TypeError: unbound method _crit_active() must be called with ActiveExplorer instance as first argument (got AdaptiveGreedy instance instead)
I copied AdaptiveExplorer._crit_active() to class AdaptiveGreedy and call
pred[set_greedy] = np.argmax(
self._crit_active(
X[set_greedy],
pred_all[set_greedy],
self.active_choice),
axis = 1)
in contextualbandits/online.py line 959. Then, everything works well but I don't know whether it is correct or not.
I probably should open a new issue for this.
Thanks.
from contextualbandits.
Thanks for your time to check the problem.
I had figured it out, but I would still answer your questions before posting my solution.
- Are you using the latest version of this package? (that is: 1.7.2)
Yes, it is the latest version.- Does it happen at the beginning (right after initialization) or after having called fit/partial_fit multiple times?
I directly used the code in online_contextual_bandits.ipynb. It took place inlst_actions[model] = simulate_rounds(models[model], lst_rewards[model], lst_actions[model], X, y, batch_st, batch_end)
- Did you modify something in running the notebook?
I didn't.- Are you running this in a regular computer/server/laptop, or is it some special architechture or cluster or something?
I ran it on a workstation, the arch should be x86_64.- Which joblib, NumPy and SciPy version are you using? In case they are old, have you tried updating them? (I tried NumPy 1.15 and SciPy 1.2)
I used joblib 0.12.5, numpy 1.15.4, scipy 1.1.0.After debugging, it is caused by
def _calculate_beta_prior(nchoices): return (3.0/nchoices, 4)
in contextualbandits/utils.py line 62.
In original version, it was 3/nchoices, when nchoices=159, it turns to 0. Now, I changed it to 3.0/nchoices=0.018. Then, it works like a charm.Another thing I would like to report is that when active_choice='weighted' for AdaptiveGreedy, it would call AdaptiveExplorer._crit_active(). This would lead to another exception
TypeError: unbound method _crit_active() must be called with ActiveExplorer instance as first argument (got AdaptiveGreedy instance instead)
I copied AdaptiveExplorer._crit_active() to class AdaptiveGreedy and call
pred[set_greedy] = np.argmax( self._crit_active( X[set_greedy], pred_all[set_greedy], self.active_choice), axis = 1)
in contextualbandits/online.py line 959. Then, everything works well but I don't know whether it is correct or not.
I probably should open a new issue for this.
Thanks.
Thanks for the detailed report. It would be correct to just move the method like that. I'll try to make some slight adjustments to make it work under Python 2.7 (these are small details in which Python 2.7 and 3.4+ differ : int / int = float in 3, and the unbound method is valid in 3), although it oddly enough ran without errors for me in 2.7.15.
from contextualbandits.
As Python 2.7 will be deprecated on Jan 2021 (if I am not mistaken), it would be fine to support Python 3.x only.
Your work "Adapting multi-armed bandits policies to contextual bandits scenarios" is interesting to me, do you plan to extend it to work with PyTorch or TensorFlow to increase its visibility?
from contextualbandits.
Updated the potential integer divisions and unbound method calls (in online
at least) - should work fine under Python 2.7 now. I don't plan to extend it to work with frameworks like tensorflow or pytorch, but if you want to use it with them, you'd just need to make them into a scikit-learn-like class with fit
/ predict
(save for a few methods that use e.g. gradients) - I've tested it with xgboost for example and works fine too.
from contextualbandits.
Related Issues (20)
- Question regarding using contextual bandits for Learning-To-Rank HOT 1
- Different predictions of the same model.dill file in different CPUs for the LinUCB algorithm HOT 1
- Support for continuous rewards HOT 1
- ParametricTS fails with: '_OneVsRest' object has no attribute 'beta_counters' HOT 2
- XGBClassifier becomes un-serializable after being used as a base_model HOT 2
- Need more understanding for the method beta_prior HOT 1
- Any good strategies to run Simulation other then one in Online policy example HOT 1
- How to deal with this Scenario while applying CB techniques HOT 2
- Getting topN arm features in Offpolicy method HOT 4
- Type error if beta_prior == "auto" and nchoices is list HOT 1
- Possibly unexpected behaviour of decision function HOT 1
- _BasePolicy.add_arm(); NameError: name 'base_algorithm' is not defined HOT 1
- AssertionError: online_contextual_bandits.ipynb HOT 1
- TypeError: contextual bandits with custom 'choice_names' (online.py) HOT 3
- Getting topN arm features in onlinepolicy method HOT 9
- topN inputs clarification HOT 3
- warm_start for online policy HOT 1
- Doubt in DREstimator HOT 3
- New release HOT 1
- If r != 0 then could be working for negative reward as well? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from contextualbandits.