Dimensionality problem with FluctuationsTwoAsset.ipynb about bayerluetticke

[llorracc]

@kossacoff, thanks for bringing this to our attention.

These things typically happen because of subtle incompatibilities in requirements (like python 3.7 vs 3.8).

@camriddell is working on trying to fix these kinds of problems right now. He's away for a couple of weeks but perhaps can look into this when he returns.

[llorracc]

@kossacoff,

Did you make sure to create an environment that has the right versions of the underlying software?

You need to do pip install -r requirements.txt and you need to make sure your version of python is the right one.

I've just rerun both the OneAsset and TwoAsset models and was able to get them to work, with the right environment (@camriddell, I can export the environment for you).

This repo is scheduled to become one of our official REMARKs soon, which should launch in a preconfigured environment, as a way of getting around these kinds of configuration problems.

[kossacoff]

Hi, @llorracc. Thanks for your answers.

I'm not so fluent in Python yet, but yesterday I learned a lot about environments, trying to create one as you recommended. In fact, I create an environment with Python 3.6.12 (following runtime.txt, is this the correct Python version?) and install the requirements in requirements.txt. Then I exported to my notebook. But I still have the same problem if I run the code with the sequential SGU_solver; now, if I run the alternative solver function, the problem appears in the multiprocessing modul/library. I think the problem is in these lines:

Inside [line 1062] the for bl in range(0, blocks): loop...

    range_= range(bl*packagesize, min(packagesize*(bl+1), mpar['numstates']))
    cc = np.zeros((mpar['numcontrols'], 1))
    ss = np.zeros((mpar['numstates'], 1))
    p1 = Process(target = FF_1_3, args = (range_, Xss, Yss, Gamma_state, indexMUdct, indexVKdct, par, mpar, grid, targets, Copula, P_H, aggrshock, Fb, packagesize, bl, ss, cc, out_DF1,  out_DF3, out_bl))
    procs1.append(p1)
    p1.start()
    print(f'Block number: {str(bl)}')    

It returns the following error: BrokenPipeError: [Errno 32] Broken pipe.

Any idea how to solve it? Thanks again!

[llorracc]

I'm guessing you are running on a Windows machine (not Mac or linux)?

See if you can figure out how to fix it by looking at what you get when you Google "BrokenPipeError: [Errno 32] Broken pipe python windows"

[kossacoff]

Yes, I'm running on a Windows. Yesterday I've been searching about that but I couldn't find something helpful, I think I'll continue looking how to solve it... by the way, I saw there is a .gitignore document in the main folder. Is it necessary to install it like the requirements text? Thanks again!

[llorracc]

@camriddell, @alanlujan91, @MridulS: Does any of you know anything about this "BrokenPipeError: [Errno 32] Broken pipe" problem that crops up when this repo is run on Windows?

PS. This is slated to become a regular REMARK, so we need to fix this problem anyway.

[kossacoff]

Finally, I could run the code on Windows writing an specific environment and using the sequential SGU solver function. The problem is still in the alternative SGU solver function, in the multiprocessing module, since the Process class is used. I found this, maybe is useful to understand how to solve the problem:

• Broken Pipe Error in Python
• Multiprocessing module

Hopefully, I think is enough to create the correct environment and use the sequential solver if this repo is run on Windows.

Thank you very much for your time and help, @llorracc!

[llorracc]

@kossacoff, delighted to hear you got it running!

Could you help us out by giving an explicit code snippet for creating the proper environment for Windows?

[llorracc]

P.S. It might run without modification if you were to use the Windows subsystem for linux for everything. Or if you were to run it in a VirtualBox virtual machine as described in econ-ark-tools

[kossacoff]

Sure. I sequentially run the following lines in the Conda prompt before running the codes in Jupyter Notebook:

conda create --name blenv python=3.6.12 -y
conda activate blenv
conda install pip
pip install -r C:...\requirements.txt
conda env export > \Users...\blenv.yaml
pip install --user ipykernel
python -m ipykernel install --user --name=blenv

Once I had the kernel available in Jupyter, I changed the default kernel to the one generated above for all the codes and run.

[llorracc]

Thanks, I've confirmed that your steps work on my own captive Windows machine that I keep for testing such things.

@camriddell, or @MridulS or @alanlujan91, how do we need to change the requirements.txt file (or what do we need to replace it with) to robustify this REMARK?