GithubHelp home page GithubHelp logo

assay's People

Contributors

brandon-rhodes avatar dkua avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar

Forkers

dkua wesleychai mmulich nandoflorestan bnavigator dusty-phillips awesomo4000

assay's Issues

Create basic README to orient the curious

Now that a few people are asking about Assay because of my talk about it at PuPPy in Seattle last week, I should add a basic README to the project explaining what works, what does not work, and maybe laying out a few near-term goals in case anyone wants to help.

Testing skyfield 1.25 with current assay fails before starting to run tests on ARM and PowerPC

Different failure than #7:

https://build.opensuse.org/package/show/devel:languages:python:numeric/python-skyfield

[   52s] + /usr/bin/python3 -m assay --batch skyfield.tests
[   52s] Traceback (most recent call last):
[   52s]   File "/usr/lib64/python3.8/runpy.py", line 194, in _run_module_as_main
[   52s]     return _run_code(code, main_globals, None,
[   52s]   File "/usr/lib64/python3.8/runpy.py", line 87, in _run_code
[   52s]     exec(code, run_globals)
[   52s]   File "/home/abuild/rpmbuild/BUILD/assay-79f5d784a55d70f31b2c90b636910738cd74840e/assay/__main__.py", line 2, in <module>
[   52s]     main()
[   52s]   File "/home/abuild/rpmbuild/BUILD/assay-79f5d784a55d70f31b2c90b636910738cd74840e/assay/command.py", line 22, in main
[   52s]     monitor.main_loop(args.name, args.batch or not isatty)
[   52s]   File "/home/abuild/rpmbuild/BUILD/assay-79f5d784a55d70f31b2c90b636910738cd74840e/assay/monitor.py", line 62, in main_loop
[   52s]     next(runner)
[   52s]   File "/home/abuild/rpmbuild/BUILD/assay-79f5d784a55d70f31b2c90b636910738cd74840e/assay/monitor.py", line 114, in runner_coroutine
[   52s]     worker = workers[0]
[   52s] IndexError: list index out of range
[   52s] error: Bad exit status from /var/tmp/rpm-tmp.X4PdOv (%check)

armv7l_log.txt (32bit!, e.g. Raspi)
aarch64_log.txt (e.g. newer Raspi)
ppc64_log.txt

Sporadic pickling errors

I get sporadic pickling errors on the openSUSE package for skyfield. Is there any way to stabilize this?

[   36s] + /usr/bin/python3 -m assay --batch skyfield.tests
[   42s] ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................Traceback (most recent call last):
[   42s]   File "/usr/lib64/python3.8/runpy.py", line 194, in _run_module_as_main
[   42s]     return _run_code(code, main_globals, None,
[   42s]   File "/usr/lib64/python3.8/runpy.py", line 87, in _run_code
[   42s]     exec(code, run_globals)
[   42s]   File "/home/abuild/rpmbuild/BUILD/assay-23c18c2457c035996057144e1fe74cd6e19b44eb/assay/__main__.py", line 2, in <module>
[   42s]     main()
[   42s]   File "/home/abuild/rpmbuild/BUILD/assay-23c18c2457c035996057144e1fe74cd6e19b44eb/assay/command.py", line 27, in main
[   42s]     monitor.main_loop(args.name, args.batch or not isatty)
[   42s]   File "/home/abuild/rpmbuild/BUILD/assay-23c18c2457c035996057144e1fe74cd6e19b44eb/assay/monitor.py", line 68, in main_loop
[   42s]     runner.send(source)
[   42s]   File "/home/abuild/rpmbuild/BUILD/assay-23c18c2457c035996057144e1fe74cd6e19b44eb/assay/monitor.py", line 144, in runner_coroutine
[   42s]     give_work_to(worker)
[   42s]   File "/home/abuild/rpmbuild/BUILD/assay-23c18c2457c035996057144e1fe74cd6e19b44eb/assay/monitor.py", line 129, in give_work_to
[   42s]     paths = [path for name, path in worker.call(list_module_paths)]
[   42s]   File "/home/abuild/rpmbuild/BUILD/assay-23c18c2457c035996057144e1fe74cd6e19b44eb/assay/worker.py", line 77, in call
[   42s]     return pickle.load(self.from_worker)
[   42s] _pickle.UnpicklingError: pickle data was truncated
[   42s] error: Bad exit status from /var/tmp/rpm-tmp.uZ8Wea (%check)

Build history for aarch64

"failed" is above error, "unchanged" and "succeeded" is passing the unit tests. Happens on other architectures too.

Testing skyfield 1.25 with current assay fails on s390x

Obscure IBM archictecture:

s390x_log.txt

[   22s] + /usr/bin/python3 -m assay --batch skyfield.tests
[   26s] .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................Traceback (most recent call last):
[   26s]   File "/usr/lib64/python3.8/runpy.py", line 194, in _run_module_as_main
[   26s]     return _run_code(code, main_globals, None,
[   26s]   File "/usr/lib64/python3.8/runpy.py", line 87, in _run_code
[   26s]     exec(code, run_globals)
[   26s]   File "/home/abuild/rpmbuild/BUILD/assay-79f5d784a55d70f31b2c90b636910738cd74840e/assay/__main__.py", line 2, in <module>
[   26s]     main()
[   26s]   File "/home/abuild/rpmbuild/BUILD/assay-79f5d784a55d70f31b2c90b636910738cd74840e/assay/command.py", line 22, in main
[   26s]     monitor.main_loop(args.name, args.batch or not isatty)
[   26s]   File "/home/abuild/rpmbuild/BUILD/assay-79f5d784a55d70f31b2c90b636910738cd74840e/assay/monitor.py", line 68, in main_loop
[   26s]     runner.send(source)
[   26s]   File "/home/abuild/rpmbuild/BUILD/assay-79f5d784a55d70f31b2c90b636910738cd74840e/assay/monitor.py", line 144, in runner_coroutine
[   26s]     give_work_to(worker)
[   26s]   File "/home/abuild/rpmbuild/BUILD/assay-79f5d784a55d70f31b2c90b636910738cd74840e/assay/monitor.py", line 129, in give_work_to
[   26s]     paths = [path for name, path in worker.call(list_module_paths)]
[   26s]   File "/home/abuild/rpmbuild/BUILD/assay-79f5d784a55d70f31b2c90b636910738cd74840e/assay/worker.py", line 77, in call
[   26s]     return pickle.load(self.from_worker)
[   26s] _pickle.UnpicklingError: pickle data was truncated
[   26s] error: Bad exit status from /var/tmp/rpm-tmp.yBjsJt (%check)

inotify not available on MacOS

Split out from discussion in #4 which is specific to polling the subprocesses to summarize my research so far.

The inotify events used in Filesystem are not supported on MacOS. Instead, we would need to use either a polling mechanism (os.scandir isn't "slow", but it's probably enough milliseconds to defeat Assay's emphasis on speed) or the C-level MacOS FSEvent polling mechanism.

There are third-party wrappers of FSEvent in Python, all of which include some sort of C extension, and most of which provide an abstraction over cross-platform filesystem monitoring. The most-supported one appears to be watchdog.

FSEvent uses a callback based approach to process events instead of returning a file descriptor that can be selected on by kqueue the way inotify does with epoll. This means that listening for fsevent changes will be fundamentally different from the other events that are happening in the monitor's main loop. (This leads me to wonder whether to refactor to let asyncio control the loop).

As an aside, one other place I noticed incorrect behaviour under macos is in unix.cpu_count() because /proc is a Linux-only thing. Perhaps it could be replaced with multiprocessing.cpu_count()?

No license file

If this gets a release, it also needs a license file in order to make it redistributable as a package. setup.py specifies MIT.

assay/setup.py

Line 8 in 79f5d78

license='MIT',

Windows compatibility

I've been using assay in linux on a virtual machine, and I really like it!

I know making it support windows is not at all trivial, but in case someday you decide that's something you want to do I'll put the error I get when trying to run it here:

Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\Scripts\assay-script.py", line 11, in <module>
    load_entry_point('assay==0.0', 'console_scripts', 'assay')()
  File "C:\ProgramData\Anaconda3\lib\site-packages\pkg_resources\__init__.py", line 570, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "C:\ProgramData\Anaconda3\lib\site-packages\pkg_resources\__init__.py", line 2751, in load_entry_point
    return ep.load()
  File "C:\ProgramData\Anaconda3\lib\site-packages\pkg_resources\__init__.py", line 2405, in load
    return self.resolve()
  File "C:\ProgramData\Anaconda3\lib\site-packages\pkg_resources\__init__.py", line 2411, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "C:\ProgramData\Anaconda3\lib\site-packages\assay\command.py", line 8, in <module>
    from . import monitor, unix
  File "C:\ProgramData\Anaconda3\lib\site-packages\assay\monitor.py", line 7, in <module>
    from . import unix
  File "C:\ProgramData\Anaconda3\lib\site-packages\assay\unix.py", line 4, in <module>
    import fcntl
ModuleNotFoundError: No module named 'fcntl'

Thanks!

assay on Python 3.8.3 fails testing skyfield

Trying skyfielders/python-skyfield#405 (comment)

In an interactive session:

abuild@greinerT450s:~/rpmbuild/BUILD/skyfield-1.24> export PYTHONPATH=../assay-18f320237345813a49173657e7d2d6ca85e9a38a
abuild@greinerT450s:~/rpmbuild/BUILD/skyfield-1.24> python3 -m assay skyfield/tests/Traceback (most recent call last):
  File "/usr/lib64/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib64/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/abuild/rpmbuild/BUILD/assay-18f320237345813a49173657e7d2d6ca85e9a38a/assay/__main__.py", line 2, in <module>
    main()
  File "/home/abuild/rpmbuild/BUILD/assay-18f320237345813a49173657e7d2d6ca85e9a38a/assay/command.py", line 22, in main
    monitor.main_loop(args.name, args.batch or not isatty)
  File "/home/abuild/rpmbuild/BUILD/assay-18f320237345813a49173657e7d2d6ca85e9a38a/assay/monitor.py", line 45, in main_loop
    file_watcher.add_paths(main_process_paths)
  File "/home/abuild/rpmbuild/BUILD/assay-18f320237345813a49173657e7d2d6ca85e9a38a/assay/filesystem.py", line 80, in add_paths
    paths = set(dirname(path) for path in file_paths) - self.paths
  File "/home/abuild/rpmbuild/BUILD/assay-18f320237345813a49173657e7d2d6ca85e9a38a/assay/filesystem.py", line 80, in <genexpr>
    paths = set(dirname(path) for path in file_paths) - self.paths
  File "/usr/lib64/python3.8/posixpath.py", line 152, in dirname
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

With --batch:

abuild@greinerT450s:~/rpmbuild/BUILD/skyfield-1.24> python3 -m assay --batch skyfield/tests/
...................................................................................................................................................................................................................................................................................................................................................................................................................................................................Traceback (most recent call last):
  File "/usr/lib64/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib64/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/abuild/rpmbuild/BUILD/assay-18f320237345813a49173657e7d2d6ca85e9a38a/assay/worker.py", line 139, in <module>
    worker_process(int(sys.argv[1]), int(sys.argv[2]), int(sys.argv[3]))
  File "/home/abuild/rpmbuild/BUILD/assay-18f320237345813a49173657e7d2d6ca85e9a38a/assay/worker.py", line 130, in worker_process
    for item in result:
  File "/home/abuild/rpmbuild/BUILD/assay-18f320237345813a49173657e7d2d6ca85e9a38a/assay/runner.py", line 35, in capture_stdout_stderr
.    for item in generator(*args):
  File "/home/abuild/rpmbuild/BUILD/assay-18f320237345813a49173657e7d2d6ca85e9a38a/assay/runner.py", line 62, in run_tests_of
    for result in run_test(module, test):
  File "/home/abuild/rpmbuild/BUILD/assay-18f320237345813a49173657e7d2d6ca85e9a38a/assay/runner.py", line 69, in run_test
    yield run_test_with_arguments(test, ())
  File "/home/abuild/rpmbuild/BUILD/assay-18f320237345813a49173657e7d2d6ca85e9a38a/assay/runner.py", line 147, in run_test_with_arguments
    rewrite_asserts_in(function)
  File "/home/abuild/rpmbuild/BUILD/assay-18f320237345813a49173657e7d2d6ca85e9a38a/assay/assertion.py", line 156, in rewrite_asserts_in
    set_code(function, types.CodeType(*args))
TypeError: an integer is required (got type bytes)
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/assay-18f320237345813a49173657e7d2d6ca85e9a38a/assay/monitor.py", line 64, in main_loop
    for source, flags in poller.events():
  File "/home/abuild/rpmbuild/BUILD/assay-18f320237345813a49173657e7d2d6ca85e9a38a/assay/unix.py", line 97, in events
    for fd, flags in self.poller.poll():

Process is stuck waiting on keyboard input. Have to KeyboardInterrupt interrupt it.

Using setup.py install does not find the entry_points in distutils and fails to install the executable script.

Python 3.11 errors in skyfield tests

There is no dis.POP_JUMP_IF_TRUE in Python 3.11 anymore. (Compare https://docs.python.org/3.11/library/dis.html vs https://docs.python.org/3.10/library/dis.html)

[    8s] + python3.10 -m assay --batch skyfield.tests
[    9s] ..........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
[    9s] 
[    9s] All 554 tests passed in 1.14 seconds

[    9s] + python3.11 -m assay --batch skyfield.tests
[    9s] Traceback (most recent call last):
[    9s]   File "<frozen runpy>", line 198, in _run_module_as_main
[    9s]   File "<frozen runpy>", line 88, in _run_code
[    9s]   File "/home/abuild/rpmbuild/BUILD/assay-bb62d1f7d51d798b05a88045fff3a2ff92c299c3/assay/__main__.py", line 1, in <module>
[    9s]     from assay.command import main
[    9s]   File "/home/abuild/rpmbuild/BUILD/assay-bb62d1f7d51d798b05a88045fff3a2ff92c299c3/assay/command.py", line 8, in <module>
[    9s]     from . import monitor, unix
[    9s]   File "/home/abuild/rpmbuild/BUILD/assay-bb62d1f7d51d798b05a88045fff3a2ff92c299c3/assay/monitor.py", line 12, in <module>
[    9s]     from .runner import capture_stdout_stderr, run_tests_of
[    9s]   File "/home/abuild/rpmbuild/BUILD/assay-bb62d1f7d51d798b05a88045fff3a2ff92c299c3/assay/runner.py", line 11, in <module>
[    9s]     from .assertion import get_code, search_for_function, rewrite_asserts_in
[    9s]   File "/home/abuild/rpmbuild/BUILD/assay-bb62d1f7d51d798b05a88045fff3a2ff92c299c3/assay/assertion.py", line 122, in <module>
[    9s]     op.pop_jump_if_true, b'.',
[    9s]     ^^^^^^^^^^^^^^^^^^^
[    9s] AttributeError: type object 'op' has no attribute 'pop_jump_if_true'

Error while running Skyfield tests

I am getting this traceback while trying to run the tests for skyfield. I'm using a bash shell inside of Windows Subsystem for Linux, but I'm hoping that's not causing the issue, because it doesn't seem like that would be easy to fix 😬. I reinstalled Assay from master to be certain I'm not missing any recent changes.

skyfield/tests/test_vectors.py line 8 in test_bad_addition
    planets = load('de421.bsp')
  skyfield/iokit.py line 182 in __call__
    return opener(path)
  skyfield/jpllib.py line 67 in __init__
    self.spk = SPK.open(path)
  /home/josh/.local/lib/python3.6/site-packages/jplephem/spk.py line 49 in open
    return cls(DAF(open(path, 'rb')))
  /home/josh/.local/lib/python3.6/site-packages/jplephem/daf.py line 62 in
  __init__
    .format(self.locidw))
ValueError: file starts with b'CI/DE421', not "NAIF/DAF" or "DAF/"

Thanks!

select.poll not available on OS X

I've been trying to run the sky field tests, and having issues.

Though from the comments in the code, this failure seems to be expected, I just want to confirm that in my version of Python (2.7.10) on OS X (10.11.6), select.poll is not available at all, and so assay crashes:

$ assay --batch skyfield.tests
Traceback (most recent call last):
  File "/Users/richard/Projects/pipeline_refactor/skyfieldtest/bin/assay", line 9, in <module>
    load_entry_point('assay==0.0', 'console_scripts', 'assay')()
  File "/Users/richard/Projects/pipeline_refactor/assay/assay/command.py", line 22, in main
    monitor.main_loop(args.name, args.batch or not isatty)
  File "/Users/richard/Projects/pipeline_refactor/assay/assay/monitor.py", line 39, in main_loop
    poller = unix.EPoll()
  File "/Users/richard/Projects/pipeline_refactor/assay/assay/unix.py", line 77, in __init__
    self.poller = select.poll()  # TODO: does this work on OS X?
AttributeError: 'module' object has no attribute 'poll'

It seems that possible options to get this one working could be moving away from poll to select which should be supported on OS X as well as Linux (though has a few limitations), or adding in support for Kqueues

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.