socialfoundations / whynot Goto Github PK

Currently the requirements file contains packages relevant to building up the docs, but not all necessary packages to installing whynot.
I propose using the setup.py file to separate the different necessary packages:

    install_requires=[

    ],
    extras_require={
        "test": [
 
        ],
        "examples": [

        ],
        "docs": [
        ],
    }

That will allow us to:

1. delete the requirements.txt file as it will be no longer needed
2. users will now be able to choose the different pkgs they wish to install
   pip install -e . will install whynot alone.
   pip install -e ".[test, docs]". would install both whynot and the pkgs necessary for test and docs, for example.

It seems like it should be pretty straightforward!
whynot gives us the graph topology already (wn.causal_graphs.build_dynamics_graph) so we just need to model it and use a graphic library to show it.
I think I have an idea on how to use graphviz for this. But open to suggestions and other people taking this.
It's mostly a suggestion for what I perceived as a low hanging fruit.

In the GiveMeSomeCredit dataset there are three features that are very correlated to each other with Pearson's r ~ 0.98 (NumberOfTime30-59DaysPastDueNotWorse, NumberOfTimes90DaysLate, NumberOfTime60-89DaysPastDueNotWorse). This multicollinearity might cause to instability in fitting Logistic Regression model. Possible mitigation is to drop these features and include only their sum.
The model on that dataset archives the same accuracy.

I'm creating this issue to discuss my current ideas/progress on making whynot compatible with the latest version of gymnasium.

I'm thinking about erasing registration.py and instead just initializing the environment on the main file, like the examples in the petting zoo:

from tutorial3_action_masking import CustomEnvironment

from pettingzoo.test import parallel_api_test

if __name__ == "__main__":
    env = CustomEnvironment()
    parallel_api_test(env, num_cycles=1_000_000)

There's no longer support for registration. This doesn't seem really necessary? But maybe I'm missing something and we should support registration ourselves?

I'm working on a bunch of forecasting problems where performative prediction feedback loops are a challenge. I'd like to create a forecasting problem simulator in whynot. Any ideas of where I could start?

whynot is almost eligible for creating a conda package. For that we need:

• a versioning system built-in to github
• the merge of conda-forge/staged-recipes#23105 which seems complicated? I thought it'd be a simple noarch package but apparently people from pyminiracer did something wrong when creating the sdist? Or something weird.

Hi,

I just started looking into that repo and went into troubles when trying to import the package in Python. I set up a new virtual environment with only whynot and dependencies installed (Python 3.8.14). Then, when running
import whynot as wn , I get the following error:

ImportError Traceback (most recent call last)
Cell In [2], line 4
1 import matplotlib.pyplot as plt
2 import numpy as np
----> 4 import whynot as wn
5 import whynot.gym as gym
7 import scripts.utils as utils

File ~/opt/miniconda3/envs/testenv/lib/python3.8/site-packages/whynot/init.py:5
3 version = "0.12.0"
4 from whynot.algorithms import *
----> 5 from whynot.simulators import *
6 from whynot import causal_graphs, dynamics, framework
7 from whynot.dynamics import (
8 DynamicsExperiment,
9 Run,
10 )

File ~/opt/miniconda3/envs/testenv/lib/python3.8/site-packages/whynot/simulators/init.py:4
1 """Simulator initialization."""
3 from whynot.simulators import civil_violence
----> 4 from whynot.simulators import credit
5 from whynot.simulators import delayed_impact
6 from whynot.simulators import dice

File ~/opt/miniconda3/envs/testenv/lib/python3.8/site-packages/whynot/simulators/credit/init.py:15
3 from whynot.simulators.credit.simulator import (
4 agent_model,
5 Config,
(...)
10 State,
11 )
13 from whynot.simulators.credit.dataloader import CreditData
---> 15 from whynot.simulators.credit.environments import *
17 SUPPORTS_CAUSAL_GRAPHS = True

File ~/opt/miniconda3/envs/testenv/lib/python3.8/site-packages/whynot/simulators/credit/environments.py:4
1 """Interactive environment for the credit simulator."""
2 import numpy as np
----> 4 from whynot.gym import spaces
5 from whynot.gym.envs import ODEEnvBuilder, register
6 from whynot.simulators.credit import (
7 agent_model,
8 CreditData,
(...)
13 State,
14 )

File ~/opt/miniconda3/envs/testenv/lib/python3.8/site-packages/whynot/gym/init.py:12
9 from gym.core import Env
10 from gym import logger
---> 12 from whynot.gym.envs import make, spec, register
14 all = ["Env", "make", "spec", "register"]

File ~/opt/miniconda3/envs/testenv/lib/python3.8/site-packages/whynot/gym/envs/init.py:3
1 """Environments based on WhyNot simulators."""
----> 3 from whynot.gym.envs.registration import registry, register, make, spec
4 from whynot.gym.envs.ode_env import ODEEnvBuilder

File ~/opt/miniconda3/envs/testenv/lib/python3.8/site-packages/whynot/gym/envs/registration.py:4
1 """Global registry of environments, for consistency with openai gym."""
2 import importlib
----> 4 from gym.envs.registration import EnvRegistry
6 # Keep for consistency with original API
7 # pylint:disable-msg=invalid-name
8 # Have a global registry
9 registry = EnvRegistry()

ImportError: cannot import name 'EnvRegistry' from 'gym.envs.registration' (/Users/.../opt/miniconda3/envs/testenv/lib/python3.8/site-packages/gym/envs/registration.py)

Because this seems to be related to gym (by default version 0.26.2 got installed), I tried out different gym versions and indeed with gym version 0.23.0 or lower the import works. So maybe, a quick-fix could be to restrict the gym version. It seems that from version 0.24.0 onwards, the gym registry module/API got changed causing an issue when importing whynot.

Having gym 0.23.0 installed, allows me to run the performative prediction example, which is what I'm currently interested in. However, in the walkthrough example there still occurs an error when running the simulator of world3:

dataset = experiment.run(num_samples=200, seed=1234, show_progress=True)

RuntimeError: Native library not available at /Users/mgorecki/opt/miniconda3/envs/testenv/lib/python3.8/site-packages/py_mini_racer/libmini_racer.dylib

I couldn't resolve that so far.

I'd like to create a multiple instances of one environment (e.g., Credit) with different configurations (e.g., each one of them with only subset of the data).

How can I do that? It seems that gym.make doesn't support that, so even creating my own modified version of a simulator won't work (except of creating multiple sub-packages, as many as different configuration I want)

A few tests are broken. Here's a list:

================================== short test summary info ===================================
FAILED tests/test_delayed_impact.py::test_repayment_fcn - TypeError: Index.get_loc() got an unexpected keyword argument 'method'
FAILED tests/test_dynamics.py::test_basestate - AssertionError: assert 4 == 3
FAILED tests/test_simulators.py::test_dynamics_initial_state[delayed_impact] - TypeError: Index.get_loc() got an unexpected keyword argument 'method'
FAILED tests/test_simulators.py::test_dynamics_initial_state[credit] - IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed
FAILED tests/test_simulators.py::test_dynamics_initial_state[world3] - TypeError: 'JSObject' object is not iterable
FAILED tests/test_simulators.py::test_dynamics_intervention[delayed_impact] - TypeError: Index.get_loc() got an unexpected keyword argument 'method'
FAILED tests/test_simulators.py::test_dynamics_intervention[world3] - TypeError: 'JSObject' object is not iterable
FAILED tests/test_simulators.py::test_simulator_experiments[delayed_impact] - TypeError: Index.get_loc() got an unexpected keyword argument 'method'
FAILED tests/test_simulators.py::test_simulator_experiments[world3] - TypeError: 'JSObject' object is not iterable
FAILED tests/test_simulators.py::test_parallelize[delayed_impact] - TypeError: Index.get_loc() got an unexpected keyword argument 'method'
FAILED tests/test_simulators.py::test_parallelize[world3] - TypeError: 'JSObject' object is not iterable
FAILED tests/test_world3.py::test_set_state - TypeError: 'JSObject' object is not iterable
FAILED tests/gym_tests/test_envs.py::test_env[spec0] - IndexError: tuple index out of range
FAILED tests/gym_tests/test_envs.py::test_env[spec1] - TypeError: RandomNumberGenerator._generator_ctor() takes from 0 to 1 positional arguments...
FAILED tests/gym_tests/test_envs.py::test_env[spec2] - TypeError: RandomNumberGenerator._generator_ctor() takes from 0 to 1 positional arguments...
FAILED tests/gym_tests/test_envs.py::test_env[spec3] - TypeError: RandomNumberGenerator._generator_ctor() takes from 0 to 1 positional arguments...
FAILED tests/gym_tests/test_envs.py::test_env[spec4] - TypeError: RandomNumberGenerator._generator_ctor() takes from 0 to 1 positional arguments...
FAILED tests/gym_tests/test_envs.py::test_random_rollout[HIV-v0] - TypeError: RandomNumberGenerator._generator_ctor() takes from 0 to 1 positional arguments...
FAILED tests/gym_tests/test_envs.py::test_random_rollout[world3-v0] - TypeError: RandomNumberGenerator._generator_ctor() takes from 0 to 1 positional arguments...
FAILED tests/gym_tests/test_envs.py::test_random_rollout[opioid-v0] - TypeError: RandomNumberGenerator._generator_ctor() takes from 0 to 1 positional arguments...
FAILED tests/gym_tests/test_envs.py::test_config[HIV-v0] - TypeError: RandomNumberGenerator._generator_ctor() takes from 0 to 1 positional arguments...
FAILED tests/gym_tests/test_envs.py::test_config[world3-v0] - TypeError: RandomNumberGenerator._generator_ctor() takes from 0 to 1 positional arguments...
FAILED tests/gym_tests/test_envs.py::test_config[opioid-v0] - TypeError: RandomNumberGenerator._generator_ctor() takes from 0 to 1 positional arguments...
FAILED tests/gym_tests/test_envs.py::test_config[Zika-v0] - TypeError: RandomNumberGenerator._generator_ctor() takes from 0 to 1 positional arguments...
FAILED tests/gym_tests/test_envs.py::test_credit_config - IndexError: tuple index out of range
FAILED tests/gym_tests/test_envs.py::test_credit_initial_state - IndexError: tuple index out of range
FAILED tests/gym_tests/test_registration.py::test_make - TypeError: RandomNumberGenerator._generator_ctor() takes from 0 to 1 positional arguments...
FAILED tests/gym_tests/test_registration.py::test_malformed_lookup - AssertionError: Unexpected message: Malformed environment ID: “Breakout-v0”.(Currently al...
=================== 28 failed, 60 passed, 56 warnings in 573.65s (0:09:33) ===================

Are these tests still relevant? Should we fix them? Did they ever run?

Hi all,

Great project!
I'd like to propose a versioning system so we can easily track changes and so users can pin this package and have it working with everything else they use (as long as other versions are pinned too).
Besides being a good practice on the open science side, it allows easy reproducibility, it's also a good software design choice and would allow whynot to be present in the conda ecosystem.
So, once we have this in place I can create a conda package for it.

The system I'd like to propose is semantic versioning, so simply v.1.0.0, if you all agree.

However before releasing a first version I'd like to merge the PR I'm writing that will fix current issues with dependencies.