Python random module is not consistent from 2 to 3. about desimodel

[sbailey]

Well, that is irritating. Python 3 looks worse and worse...

Adam and I confirmed that numpy is reproducible across python 2.7 and 3.5, e.g.

Python 2.7.12 |Anaconda 2.0.1 (x86_64)| (default, Jul  2 2016, 17:43:17) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import numpy as np
>>> np.random.seed(2)
>>> np.random.randint(100, size=20)
array([40, 15, 72, 22, 43, 82, 75,  7, 34, 49, 95, 75, 85, 47, 63, 31, 90,
       20, 37, 39])

(py3) ~ $ python
Python 3.5.2 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:52:12) 
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.random.seed(2)
>>> np.random.randint(100, size=20)
array([40, 15, 72, 22, 43, 82, 75,  7, 34, 49, 95, 75, 85, 47, 63, 31, 90,
       20, 37, 39])

Another bizarre thing with the python system library random: on my laptop, the sequence I get from ipython is one element offset from the sequence I get from vanilla python:

~ $ python
Python 2.7.12 |Anaconda 2.0.1 (x86_64)| (default, Jul  2 2016, 17:43:17) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import random; random.seed(2); random.sample(range(100), 20)
[95, 94, 5, 8, 83, 73, 66, 30, 60, 58, 15, 43, 39, 72, 99, 54, 44, 26, 3, 2]

~ $ ipython
Python 2.7.12 |Anaconda 2.0.1 (x86_64)| (default, Jul  2 2016, 17:43:17) 
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import random

In [2]: random.seed(2)

In [3]: random.sample(range(100),20)
Out[3]: [94, 5, 8, 83, 73, 66, 30, 60, 58, 15, 43, 39, 72, 99, 54, 44, 26, 3, 2, 46]

and to continue the bizarreness, I get the python sequence from ipython if I chain the commands together in a single line:

~ $ ipython
Python 2.7.12 |Anaconda 2.0.1 (x86_64)| (default, Jul  2 2016, 17:43:17) 
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import random; random.seed(2); random.sample(range(100), 20)
   ...: 
Out[1]: [95, 94, 5, 8, 83, 73, 66, 30, 60, 58, 15, 43, 39, 72, 99, 54, 44, 26, 3, 2]

Guidelines:

• don't use the python system library random
• use numpy.random instead
• this is crazy

[sbailey]

wrong button; reopening until we've confirmed that we are using numpy.random throughout.

[weaverba137]

Further weirdness. The fundamental function of the random module is random.random(); almost all the other functions depend on it. In Python 2 & 3, it does appear to be consistent.

Python 2:

Python 2.7.12 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:43:17) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import random
>>> random.seed(2)
>>> random.random()
0.9560342718892494
>>> random.random()
0.9478274870593494
>>> random.random()
0.05655136772680869
>>> random.random()
0.08487199515892163

Python 3:

Python 3.5.2 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:52:12) 
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import random
>>> random.seed(2)
>>> random.random()
0.9560342718892494
>>> random.random()
0.9478274870593494
>>> random.random()
0.05655136772680869
>>> random.random()
0.08487199515892163

So maybe for random.sample() the way the list members are hashed is what matters? Or maybe changes in how range() works? I still agree with your guidelines though!

[weaverba137]

The random.sample() algorithm requires a random integer in the range [0,n). Python 2 & 3 have different mechanisms for obtaining that random integer, and it looks like the different mechanisms call random.random() a different number of times.

[weaverba137]

Does anyone care about this issue? We're weeks away from dropping Python 2 entirely, so it doesn't seem relevant anymore.

[sbailey]

Agreed. Closing this ticket, while reiterating the guidelines:

• don't use the python system library random
• use numpy.random instead
• this is crazy