Flux executor broken about psij-python

[SteVwonder]

What are the details of the job being run? Is it asking for multiple nodes/cores or GPUs?

[andre-merzky]

test_run.py is really just running /bin/date:

#!/usr/bin/env python3

# This is meant as a simple test file to check if psij was installed successfully

import sys

from psij import Job, JobExecutor, JobSpec


if __name__ == '__main__':
    name = 'local'
    if len(sys.argv) > 1:
        name = sys.argv[1]
    jobs = list()
    job = Job(JobSpec(executable='/bin/date'))
    exec = JobExecutor.get_instance(name)
    exec.submit(job)
    jobs.append(job)
    print('Job submitted')
    status = job.wait()
    print('Job done: {}'.format(status))

The generated flux spec is:

{'attributes': {'system': {'duration': 600.0}},
 'resources': [{'count': 1,
                'label': 'task',
                'type': 'slot',
                'with': [{'count': 1, 'type': 'core'},
                         {'count': 1, 'type': 'gpu'}]}],
 'tasks': [{'command': ['/bin/date'],
            'count': {'per_slot': 1},
            'slot': 'task'}],
 'version': 1}

I actually don't understand why a GPU is requested, but either way, a GPU is available also.

Edit: Duh, the gpu is requested because of this:

            jspec.resources = ResourceSpecV1(process_count=1,
                                             processes_per_node=1,
                                             cpu_cores_per_process=1,
                                             gpu_cores_per_process=1,
                                             exclusive_node_use=True)

That looks at least incomplete / wrong. Will fix. The question remains why does the spec fail to run?

[SteVwonder]

The question remains why does the spec fail to run?

That's a good quesiton. It may be because the Flux instance you are submitting to does not include the flux-sched Fluxion scheduler. If you run flux module list is sched-simple loaded? If so, I believe that is the issue since that scheduler does not support GPUs. The other issue could be Flux not detecting the GPUs due to an hwloc that isn't configured against CUDA. Does flux resource list report any GPUs?