jsdpag / pysyncq

Python Synchronisation Queue

This is a multiprocessing data structure that coordinates a set of child processes in two senses.

1. Data - Any connected process can add data to the queue
   for all others to read.
2. Time - Processes can wait on the queue for new data to
   be added by any other.

Every process that connects to the same queue is both a reader and writer i.e. it is both a producer and a consumer. But a piece of data is not removed from the queue until every connected process has had a chance to read it. Hence, one process can signal all of the others to respond to the same information.

PyPI package at pypi.org/project/pysyncq

Online documentation at pysyncq.readthedocs.io

GitHub project at github.com/jsdpag/pysyncq

• See docs/_build/html/index.html for basic API documentation.
• See pysyncq/tests/demo.py for a simple demonstration of
  basic timing signals and message passing between processes.
• See pysyncq/tests/benchmark.py for a simple benchmarking of the message transfer time from a sender to a reader.

Developed by:

• Jackson Smith

0.0.1 - Support for spawn as well as fork child process starting methods. 0.0.0 - Initial release. Functional on Linux.

pysyncq's Issues

Windows compatability

There are at least two problems.

1. header.py uses Unix specific library resource.
2. memoryview cannot be pickled across the spawn to a child process.

Work arounds:

1. Check OS and assign hard-coded default value if Windows.
2. Check OS. Establish memoryview objects in call to .open( ) method if Windows, rather than during .__init__( ). If *nix then do establish memoryviews during init, for sharing amongst all child processes.

Default sender value in open( ) runs once during module import.

This can carry pid of parent to the child process.

Suggested solution

Set default to None, in which case the current process pid is fetched in method body.