troelsfr / batchq Goto Github PK
View Code? Open in Web Editor NEWPython automatisation tools
License: MIT License
batchq's People
Contributors
Stargazers
Watchers
batchq's Issues
Add len to Collection
Remove queue / descriptor
If possible, remove the queue, or the descriptor.
Hashing of individual files
The hashing method should support hashing of individual files and/or directories.
lazy status function
Increase performance and stability by introducing lazy_finished, lazy_running etc.
Rename base classes
BatchQ -> BaseQueue
DescriptorQ -> Descriptor
Inheritance of queus
Fix it such that first argument of the queue constructor can be a queue.
Job queuing command
The package should contain a job queuing command, i.e.
$ q lsf job@brutus,config
Uploading input directory.
Submitted job on brutus.ethz.ch
$ q lsf job@brutus,config
Job pending on brutus.ethz.ch
$ q lsf job@brutus,config
Job running on brutus.ethz.ch
$ q lsf job@brutus,config
Job finished on brutus.ethz.ch
Retrieving results.
Do you want to remove the directory 'Submission2' on brutus.ethz.ch (Y/N)? Y
Deleted Submission2 on brutus.ethz.ch
Install script
create the python setup.py for setup tools and make the package easy_installable.
Fix hashing of empty directories
When hashing empty directories the package fails
Implement reconnect
Recursion on get attribute
Traceback (most recent call last):
File "quickstart5.py", line 8, in
desc.job()
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 1054, in getattribute
fnc = getattr(self.get_queue(), name)
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 1031, in get_queue
self._queue.fields[n].set(val)
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 873, in getattribute
if hasattr(self, "fields") and name in object.getattribute(self,"fields"):
File "/Users/tfr/Documents/BatchQ/batchq/core/batch.py", line 871, in getattribute
def getattribute(self,name):
KeyboardInterrupt
export TERM for all bash and ssh sessions
global flag for configuration
Create the possibility to do stuff like:
q configuration my_server_configuration --working_directory="Submission" --command="./script" --input_directory="." --port=22 --server="server.address.com" --global=1
Store pipes in global dictionary and reuse them
DescriptorQ as function
Apparently you can call DescriptorQ as a function. This should not be possible.
Implement wall, mpi, processes, max_mem, max_space
This should be implemented for lsf for a beginning .
VisTrails package
Contribution package Vistrails which implements all queue objects as workflow units.
Support for remote PBS
I would like to add support for remote PBS. Are there any plans to do this? Do you have any pointers on where to begin?
Multi-configuration of commandline tool
Basically we want to be able to do:
$ q lsf submit@brutus,conf1
Change Collection into Q
Add following initialisation methods:
Q( Q(), [*_dict] )
Q( *_dict )
Time formatter
There should be implemented a time formatter i.e. descriptor(time = "10h5m2s" ) should be converted into descriptor(time=(10_60 + 5)_60 + 2), automatically
Long comments makes the process/terminal/communication module hang
Delete password from settings dictionaries
Password should be deleted once used.
TERM set on process spawn
The TERM shoud be set when a process is spawn using the environment variables. Possibly create a batchq.core.utils.enviroment function
Overwrite submission ID from shell
Test wether overwrite submission id from the shell works, i.e.
q job@brutus --overwrite_submission_id="your_custom_id"
Add adjust_repsonse_time to BasePipe
The implementation should not be much different from SSHTerminal.connection_lost except that the time is measured.
Introduce Q
Q is a function which produces a collection. Q takes either a list of descriptors, a single descriptor og descriptor initialisation parameters as argument.
Bjobs latency problem
There occurs a delay from submission with bsub until the job is in the bjobs list.
Unknown state once job is gone from bjobs
The q should be able to detect wether a job has finished independently of wether a job occurs in the queue or not
Default configuration files
Create a default configuration file directory which is shipped with the package
Change the __str__ behaviour of Collection
Terminal access as property and not as function
Instead of
Queue.terminal().buffer
the syntax should be
Queue.terminal.buffer
to be consistent with the way queues are defined.
Reset cache after job submission
The cache should be reset after a job submission.
Python 3 support
Current version only supports Python 2.7
Implement input_paths
support for ssh control master
To my knowledge, ssh connections that uses control master does not work. BatchQ tries to enter a password, although the session is already started.
input_files as Property
Create a queue property which take files for generating the hash instead of a directory.
Initialisation of Descriptor
The descriptor should copy the configuration dict of initialisation object i.e. DescriptorQ(Brutus() )
Fix nohup such that it matches the new structure
Terminal does not support word wrapping
The terminal does not support word wrapping. This should be fixed by since bash as standard requires this.
Stable branch does not build
Running python setup.py build results in an error:
error: file '~/code/BatchQ/batchq/bin/q' does not exist
Controllers are not initialised
Controllers are not initialised and can therefore not be used for any class instance.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.