Missing dependencies for pyspark in README about jgit-spark-connector

[ajnavarro]

Thanks for the report!

This bug is fixed on 0.7.0 version. You can check it executing: pyspark --packages "tech.sourced:engine:0.7.0"

[alejandrosame]

Maybe I'm doing something wrong, but I still need to install bblfsh through pip to make it work.

With pyspark --packages "tech.sourced:engine:0.6.4":

>>> jars = [y for (x, y) in sc.getConf().getAll() if x == 'spark.jars']
>>> print([x.split("/")[-1] for x in jars[0].split(",") if 'sourced' in x or 'bblfsh' in x])
['tech.sourced_engine-0.6.4.jar', 'tech.sourced_siva-java-0.1.3.jar', 'org.bblfsh_bblfsh-client-1.8.2.jar', 'tech.sourced_enry-java-1.6.3.jar']

>>> from sourced.engine import Engine
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/alejandro/.local/lib/python3.6/site-packages/sourced/engine/__init__.py", line 3, in <module>
    from bblfsh.sdkversion import VERSION
ModuleNotFoundError: No module named 'bblfsh'

And with pyspark --packages "tech.sourced:engine:0.7.0":

>>> jars = [y for (x, y) in sc.getConf().getAll() if x == 'spark.jars']
>>> print([x.split("/")[-1] for x in jars[0].split(",") if 'sourced' in x or 'bblfsh' in x])
['tech.sourced_engine-0.7.0.jar', 'tech.sourced_siva-java-0.1.3.jar', 'org.bblfsh_bblfsh-client-1.9.1.jar', 'tech.sourced_enry-java-1.6.3.jar']

>>> from sourced.engine import Engine
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/alejandro/.local/lib/python3.6/site-packages/sourced/engine/__init__.py", line 3, in <module>
    from bblfsh.sdkversion import VERSION
ModuleNotFoundError: No module named 'bblfsh'

[ajnavarro]

Thanks for the report @alejandrosame !, We are having a look.

[erizocosmico]

@alejandrosame Did you install the latest sourced-engine python package? It should already install bblfsh without requiring you to manually install it.

[alejandrosame]

@erizocosmico, @ajnavarro Indeed that worked! Thank you!

[erizocosmico]

Great! I will add more detailed requirements for python to the README, anyway.