PBS job templates not picked up? about ipyparallel

[paulmelis]

Well, the documentation might still need a remark added, as it doesn't show full paths to the template files.

[minrk]

Btw, are the profiles that get generated with "profile create" written based on inspecting Python classes or something?

The unstable ordering is a bug that will be fixed in traitlets 4.1.

[paulmelis]

Okay, so I was using the wrong configuration item. c.PBSControllerLauncher.batch_file_name should be c.PBSControllerLauncher.batch_template_file (and similar for the engine template).

But the template file name needs to be a full path to a file, its seems. As otherwise it won't be found. I.e. with

c.PBSControllerLauncher.batch_template_file = 'controller.template'
c.PBSEngineSetLauncher.batch_template_file = 'engine.template'

and both template files located in $HOME/.ipython/profile_1engine-per-node I get:

(3.4) paulm@login:~$ ipcluster start --profile=1engine-per-node -n 2
2015-09-11 14:09:35.377 [IPClusterStart] Starting ipcluster with [daemon=False]
2015-09-11 14:09:35.378 [IPClusterStart] Creating pid file: /home/paulm/.ipython/profile_1engine-per-node/pid/ipcluster.pid
2015-09-11 14:09:35.379 [IPClusterStart] Starting Controller with PBS
2015-09-11 14:09:35.380 [IPClusterStart] ERROR | Controller start failed
Traceback (most recent call last):
  File "/home/paulm/pyenv/3.4/lib/python3.4/site-packages/ipyparallel/apps/ipclusterapp.py", line 503, in start_controller
    self.controller_launcher.start()
  File "/home/paulm/pyenv/3.4/lib/python3.4/site-packages/ipyparallel/apps/launcher.py", line 1188, in start
    return super(PBSControllerLauncher, self).start(1)
  File "/home/paulm/pyenv/3.4/lib/python3.4/site-packages/ipyparallel/apps/launcher.py", line 1135, in start
    self.write_batch_script(n)
  File "/home/paulm/pyenv/3.4/lib/python3.4/site-packages/ipyparallel/apps/launcher.py", line 1099, in write_batch_script
    with open(self.batch_template_file) as f:
FileNotFoundError: [Errno 2] No such file or directory: 'controller.template'
ERROR:tornado.application:Exception in callback functools.partial(<function wrap.<locals>.wrapped at 0x7f462a5de0d0>)
Traceback (most recent call last):
  File "/home/paulm/pyenv/3.4/lib/python3.4/site-packages/zmq/eventloop/minitornado/ioloop.py", line 463, in _run_callback
    callback()
  File "/home/paulm/pyenv/3.4/lib/python3.4/site-packages/zmq/eventloop/minitornado/stack_context.py", line 331, in wrapped
    raise_exc_info(exc)
  File "<string>", line 3, in raise_exc_info
  File "/home/paulm/pyenv/3.4/lib/python3.4/site-packages/zmq/eventloop/minitornado/stack_context.py", line 302, in wrapped
    ret = fn(*args, **kwargs)
  File "/home/paulm/pyenv/3.4/lib/python3.4/site-packages/ipyparallel/apps/ipclusterapp.py", line 548, in start
    self.start_controller()
  File "/home/paulm/pyenv/3.4/lib/python3.4/site-packages/ipyparallel/apps/ipclusterapp.py", line 503, in start_controller
    self.controller_launcher.start()
  File "/home/paulm/pyenv/3.4/lib/python3.4/site-packages/ipyparallel/apps/launcher.py", line 1188, in start
    return super(PBSControllerLauncher, self).start(1)
  File "/home/paulm/pyenv/3.4/lib/python3.4/site-packages/ipyparallel/apps/launcher.py", line 1135, in start
    self.write_batch_script(n)
  File "/home/paulm/pyenv/3.4/lib/python3.4/site-packages/ipyparallel/apps/launcher.py", line 1099, in write_batch_script
    with open(self.batch_template_file) as f:
FileNotFoundError: [Errno 2] No such file or directory: 'controller.template'

[minrk]

And if you specify them as full paths?

[paulmelis]

Well, yes, that's what I wrote. I need to specify their full paths, and then they are found.

[minrk]

Then this issue can be closed?

[minrk]

Hi! I’m going through and cleaning up old/stale issues on this repo. Sorry for leaving it forever. I'm hoping to bring this repo back to a healthier state.

Template paths are evaluated relative to the cwd, like most file path arguments (e.g. cat pbs.engine.template). Absolute paths always work, and relative paths are not treated specially, but are sensitive to the working directory.