should pkg_resources be a dependency? about plotly.py

[theengineear]

@chriddyp : is this still a problem we need to address?

[AnneTheAgile]

@theengineear , yes, please I would like to address this. Anaconda Skeletons cannot be built. Where is this pkg_resources from? Is this related to the changes over time in the easy install, etc? I would be happy to submit a patch but just adding an import didn't work for me. This was one of the first Google hits that came up :)

[theengineear]

@AnneTheAgile , we include data in .json files that are included via this line in the setup.py:
https://github.com/plotly/python-api/blob/master/setup.py#L41

package_data={'plotly': ['graph_reference/*.json']}

To grab this data from a module we're doing this:
https://github.com/plotly/python-api/blob/master/plotly/graph_objs/graph_objs_tools.py#L14-36

It would be awesome if you wanted to submit a patch. You can include data (non-.py files) in various other ways. It would be great if you found a way that could work more universally.

[AnneTheAgile]

@theengineear , ok cool. I did a little research and it seems like the problem could be versions of distribute vs setuptools. I'm guessing a bit since I don't know the internals yet.

Here is what I'm seeing. Conda can't build your setup or rather, what it interpreted your setup to be. It fails on importing package_data.

## Nano on the file shows this is the line that fails; 
from pkg_resources import resource_string

## The very long command output is as follows:
# conda-build == conda build
$ sudo conda-build plotly
Password:
BUILD START: plotly-1.2.9-py27_0
Fetching package metadata: .....
Solving package specifications: .
The following NEW packages will be INSTALLED:

    openssl:    1.0.1h-1
    python:     2.7.8-1
    readline:   6.2-2
    requests:   2.4.3-py27_0
    setuptools: 5.8-py27_0
    six:        1.8.0-py27_0
    sqlite:     3.8.4.1-0
    tk:         8.5.15-0
    zlib:       1.2.7-1

Linking packages ...
[      COMPLETE      ] |##################################################| 100%
Source cache directory is: /Users/myuser/anaconda/conda-bld/src_cache
Found source in cache: plotly-1.2.9.tar.gz
Package: plotly-1.2.9-py27_0
source tree in: /Users/myuser/anaconda/conda-bld/work/plotly-1.2.9
+ /Users/myuser/anaconda/envs/_build/bin/python setup.py install
running install
running bdist_egg
running egg_info
writing requirements to plotly.egg-info/requires.txt
writing plotly.egg-info/PKG-INFO
writing top-level names to plotly.egg-info/top_level.txt
writing dependency_links to plotly.egg-info/dependency_links.txt
reading manifest file 'plotly.egg-info/SOURCES.txt'
writing manifest file 'plotly.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.5-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/plotly
copying plotly/__init__.py -> build/lib/plotly
copying plotly/exceptions.py -> build/lib/plotly
copying plotly/tools.py -> build/lib/plotly
copying plotly/utils.py -> build/lib/plotly
copying plotly/version.py -> build/lib/plotly
creating build/lib/plotly/plotly
copying plotly/plotly/__init__.py -> build/lib/plotly/plotly
copying plotly/plotly/plotly.py -> build/lib/plotly/plotly
creating build/lib/plotly/plotly/chunked_requests
copying plotly/plotly/chunked_requests/__init__.py -> build/lib/plotly/plotly/chunked_requests
copying plotly/plotly/chunked_requests/chunked_request.py -> build/lib/plotly/plotly/chunked_requests
creating build/lib/plotly/graph_objs
copying plotly/graph_objs/__init__.py -> build/lib/plotly/graph_objs
copying plotly/graph_objs/graph_objs.py -> build/lib/plotly/graph_objs
copying plotly/graph_objs/graph_objs_tools.py -> build/lib/plotly/graph_objs
creating build/lib/plotly/matplotlylib
copying plotly/matplotlylib/__init__.py -> build/lib/plotly/matplotlylib
copying plotly/matplotlylib/mpltools.py -> build/lib/plotly/matplotlylib
copying plotly/matplotlylib/renderer.py -> build/lib/plotly/matplotlylib
creating build/lib/plotly/matplotlylib/mplexporter
copying plotly/matplotlylib/mplexporter/__init__.py -> build/lib/plotly/matplotlylib/mplexporter
copying plotly/matplotlylib/mplexporter/_py3k_compat.py -> build/lib/plotly/matplotlylib/mplexporter
copying plotly/matplotlylib/mplexporter/exporter.py -> build/lib/plotly/matplotlylib/mplexporter
copying plotly/matplotlylib/mplexporter/tools.py -> build/lib/plotly/matplotlylib/mplexporter
copying plotly/matplotlylib/mplexporter/utils.py -> build/lib/plotly/matplotlylib/mplexporter
creating build/lib/plotly/matplotlylib/mplexporter/renderers
copying plotly/matplotlylib/mplexporter/renderers/__init__.py -> build/lib/plotly/matplotlylib/mplexporter/renderers
copying plotly/matplotlylib/mplexporter/renderers/base.py -> build/lib/plotly/matplotlylib/mplexporter/renderers
copying plotly/matplotlylib/mplexporter/renderers/fake_renderer.py -> build/lib/plotly/matplotlylib/mplexporter/renderers
copying plotly/matplotlylib/mplexporter/renderers/vega_renderer.py -> build/lib/plotly/matplotlylib/mplexporter/renderers
copying plotly/matplotlylib/mplexporter/renderers/vincent_renderer.py -> build/lib/plotly/matplotlylib/mplexporter/renderers
creating build/lib/plotly/graph_reference
copying plotly/graph_reference/graph_objs_checklist.json -> build/lib/plotly/graph_reference
copying plotly/graph_reference/graph_objs_keys.json -> build/lib/plotly/graph_reference
copying plotly/graph_reference/graph_objs_meta.json -> build/lib/plotly/graph_reference
creating build/bdist.macosx-10.5-x86_64
creating build/bdist.macosx-10.5-x86_64/egg
creating build/bdist.macosx-10.5-x86_64/egg/plotly
copying build/lib/plotly/__init__.py -> build/bdist.macosx-10.5-x86_64/egg/plotly
copying build/lib/plotly/exceptions.py -> build/bdist.macosx-10.5-x86_64/egg/plotly
creating build/bdist.macosx-10.5-x86_64/egg/plotly/graph_objs
copying build/lib/plotly/graph_objs/__init__.py -> build/bdist.macosx-10.5-x86_64/egg/plotly/graph_objs
copying build/lib/plotly/graph_objs/graph_objs.py -> build/bdist.macosx-10.5-x86_64/egg/plotly/graph_objs
copying build/lib/plotly/graph_objs/graph_objs_tools.py -> build/bdist.macosx-10.5-x86_64/egg/plotly/graph_objs
creating build/bdist.macosx-10.5-x86_64/egg/plotly/graph_reference
copying build/lib/plotly/graph_reference/graph_objs_checklist.json -> build/bdist.macosx-10.5-x86_64/egg/plotly/graph_reference
copying build/lib/plotly/graph_reference/graph_objs_keys.json -> build/bdist.macosx-10.5-x86_64/egg/plotly/graph_reference
copying build/lib/plotly/graph_reference/graph_objs_meta.json -> build/bdist.macosx-10.5-x86_64/egg/plotly/graph_reference
creating build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib
copying build/lib/plotly/matplotlylib/__init__.py -> build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib
creating build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter
copying build/lib/plotly/matplotlylib/mplexporter/__init__.py -> build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter
copying build/lib/plotly/matplotlylib/mplexporter/_py3k_compat.py -> build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter
copying build/lib/plotly/matplotlylib/mplexporter/exporter.py -> build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter
creating build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter/renderers
copying build/lib/plotly/matplotlylib/mplexporter/renderers/__init__.py -> build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter/renderers
copying build/lib/plotly/matplotlylib/mplexporter/renderers/base.py -> build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter/renderers
copying build/lib/plotly/matplotlylib/mplexporter/renderers/fake_renderer.py -> build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter/renderers
copying build/lib/plotly/matplotlylib/mplexporter/renderers/vega_renderer.py -> build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter/renderers
copying build/lib/plotly/matplotlylib/mplexporter/renderers/vincent_renderer.py -> build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter/renderers
copying build/lib/plotly/matplotlylib/mplexporter/tools.py -> build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter
copying build/lib/plotly/matplotlylib/mplexporter/utils.py -> build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter
copying build/lib/plotly/matplotlylib/mpltools.py -> build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib
copying build/lib/plotly/matplotlylib/renderer.py -> build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib
creating build/bdist.macosx-10.5-x86_64/egg/plotly/plotly
copying build/lib/plotly/plotly/__init__.py -> build/bdist.macosx-10.5-x86_64/egg/plotly/plotly
creating build/bdist.macosx-10.5-x86_64/egg/plotly/plotly/chunked_requests
copying build/lib/plotly/plotly/chunked_requests/__init__.py -> build/bdist.macosx-10.5-x86_64/egg/plotly/plotly/chunked_requests
copying build/lib/plotly/plotly/chunked_requests/chunked_request.py -> build/bdist.macosx-10.5-x86_64/egg/plotly/plotly/chunked_requests
copying build/lib/plotly/plotly/plotly.py -> build/bdist.macosx-10.5-x86_64/egg/plotly/plotly
copying build/lib/plotly/tools.py -> build/bdist.macosx-10.5-x86_64/egg/plotly
copying build/lib/plotly/utils.py -> build/bdist.macosx-10.5-x86_64/egg/plotly
copying build/lib/plotly/version.py -> build/bdist.macosx-10.5-x86_64/egg/plotly
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/__init__.py to __init__.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/exceptions.py to exceptions.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/graph_objs/__init__.py to __init__.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/graph_objs/graph_objs.py to graph_objs.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/graph_objs/graph_objs_tools.py to graph_objs_tools.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/__init__.py to __init__.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter/__init__.py to __init__.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter/_py3k_compat.py to _py3k_compat.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter/exporter.py to exporter.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter/renderers/__init__.py to __init__.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter/renderers/base.py to base.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter/renderers/fake_renderer.py to fake_renderer.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter/renderers/vega_renderer.py to vega_renderer.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter/renderers/vincent_renderer.py to vincent_renderer.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter/tools.py to tools.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mplexporter/utils.py to utils.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/mpltools.py to mpltools.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/matplotlylib/renderer.py to renderer.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/plotly/__init__.py to __init__.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/plotly/chunked_requests/__init__.py to __init__.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/plotly/chunked_requests/chunked_request.py to chunked_request.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/plotly/plotly.py to plotly.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/tools.py to tools.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/utils.py to utils.pyc
byte-compiling build/bdist.macosx-10.5-x86_64/egg/plotly/version.py to version.pyc
creating build/bdist.macosx-10.5-x86_64/egg/EGG-INFO
copying plotly.egg-info/PKG-INFO -> build/bdist.macosx-10.5-x86_64/egg/EGG-INFO
copying plotly.egg-info/SOURCES.txt -> build/bdist.macosx-10.5-x86_64/egg/EGG-INFO
copying plotly.egg-info/dependency_links.txt -> build/bdist.macosx-10.5-x86_64/egg/EGG-INFO
copying plotly.egg-info/not-zip-safe -> build/bdist.macosx-10.5-x86_64/egg/EGG-INFO
copying plotly.egg-info/requires.txt -> build/bdist.macosx-10.5-x86_64/egg/EGG-INFO
copying plotly.egg-info/top_level.txt -> build/bdist.macosx-10.5-x86_64/egg/EGG-INFO
creating dist
creating 'dist/plotly-1.2.9-py2.7.egg' and adding 'build/bdist.macosx-10.5-x86_64/egg' to it
removing 'build/bdist.macosx-10.5-x86_64/egg' (and everything under it)
Processing plotly-1.2.9-py2.7.egg
creating /Users/myuser/anaconda/envs/_build/lib/python2.7/site-packages/plotly-1.2.9-py2.7.egg
Extracting plotly-1.2.9-py2.7.egg to /Users/myuser/anaconda/envs/_build/lib/python2.7/site-packages
Adding plotly 1.2.9 to easy-install.pth file

Installed /Users/myuser/anaconda/envs/_build/lib/python2.7/site-packages/plotly-1.2.9-py2.7.egg
Processing dependencies for plotly==1.2.9
Searching for six==1.8.0
Best match: six 1.8.0
Adding six 1.8.0 to easy-install.pth file

Using /Users/myuser/anaconda/envs/_build/lib/python2.7/site-packages
Searching for requests==2.4.3
Best match: requests 2.4.3
Adding requests 2.4.3 to easy-install.pth file

Using /Users/myuser/anaconda/envs/_build/lib/python2.7/site-packages
Finished processing dependencies for plotly==1.2.9
found egg dir: /Users/myuser/anaconda/envs/_build/lib/python2.7/site-packages/plotly-1.2.9-py2.7.egg
found egg: /Users/myuser/anaconda/envs/_build/lib/python2.7/site-packages/setuptools-5.8-py2.7.egg
number of files: 54
BUILD END: plotly-1.2.9-py27_0
TEST START: plotly-1.2.9-py27_0
Fetching package metadata: .....
Solving package specifications: .
The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    plotly-1.2.9               |           py27_0         106 KB

The following NEW packages will be INSTALLED:

    openssl:  1.0.1h-1
    plotly:   1.2.9-py27_0
    python:   2.7.8-1
    readline: 6.2-2
    requests: 2.4.3-py27_0
    six:      1.8.0-py27_0
    sqlite:   3.8.4.1-0
    tk:       8.5.15-0
    zlib:     1.2.7-1

Fetching packages ...
plotly-1.2.9-p 100% |################################| Time: 0:00:00  46.41 MB/s
Extracting packages ...
[      COMPLETE      ] |##################################################| 100%
Linking packages ...
[      COMPLETE      ] |##################################################| 100%
===== testing package: plotly-1.2.9-py27_0 =====
import: u'plotly'
Traceback (most recent call last):
  File "/Users/myuser/anaconda/conda-bld/test-tmp_dir/run_test.py", line 25, in <module>
    import plotly
  File "/Users/myuser/anaconda/envs/_test/lib/python2.7/site-packages/plotly/__init__.py", line 31, in <module>
    from plotly import plotly, graph_objs, tools, utils
  File "/Users/myuser/anaconda/envs/_test/lib/python2.7/site-packages/plotly/plotly/__init__.py", line 10, in <module>
    from . plotly import (
  File "/Users/myuser/anaconda/envs/_test/lib/python2.7/site-packages/plotly/plotly/plotly.py", line 29, in <module>
    from plotly import tools
  File "/Users/myuser/anaconda/envs/_test/lib/python2.7/site-packages/plotly/tools.py", line 21, in <module>
    from . graph_objs import graph_objs
  File "/Users/myuser/anaconda/envs/_test/lib/python2.7/site-packages/plotly/graph_objs/__init__.py", line 12, in <module>
    from . graph_objs import (
  File "/Users/myuser/anaconda/envs/_test/lib/python2.7/site-packages/plotly/graph_objs/graph_objs.py", line 48, in <module>
    from pkg_resources import resource_string
ImportError: No module named pkg_resources
TESTS FAILED: plotly-1.2.9-py27_0

Doing that works fine in my environment generally.

Python 2.7.6 (v2.7.6:3a1db0d2747e, Nov 10 2013, 00:42:54)
>>> from pkg_resources import resource_string
>>> quit()

So I'm wondering, could you tell me the specific versions of package_data and setuptools / dist you are using?

[theengineear]

@AnneTheAgile , here's my setuptools:

ee@newb:~$ easy_install --version
setuptools 5.0.1
ee@newb:~$ 

I've wondered if we can circumvent this completely using the manifest.in template:
https://docs.python.org/2/distutils/sourcedist.html#manifest-template

I've only used the package_data in setup.py, so I'm not as familiar, but it seems straightforward enough. That might also solve your problem?

[theengineear]

We no longer use pkg_resources. Hopefully, related issues will be fixed at this point!