specification of default configs for notebooks created through the web UI about determined

[vishnu2kmohan]

Just a note that we're going to start working on this effort soon @zjorgensenbits, and will update this issue when it's ready.

[zjorgensenbits]

I will check it out when we update to this version and let you know. But, based on the screenshot, this is exactly what I had in mind!

[neilconway]

@zjorgensenbits Thanks for writing this up! I agree that this would be a useful feature. We've started thinking about the best way to implement it -- will keep you posted if we have any updates on the timeline here.

[zjorgensenbits]

Any thoughts/progress on this?

[vishnu2kmohan]

Update: We'll add the ability to pick a configuration template to the notebook launch dialog and post an update once this feature ships.

[zjorgensenbits]

That would be super helpful, thanks!

[hamidzr]

Hi, @zjorgensenbits and thanks for sharing your feedback. We recently shipped a new and more configurable way of launching Jupyeter Notebooks that allows you to pick a template for the Notebook configuration you're running. Does this address what you've expressed?

[vishnu2kmohan]

Hey @zjorgensenbits have you had a chance to upgrade and test 0.16.x out?

Any feedback?

And if it's what you and/or your team were after, may we please close out this issue?

[hamidzr]

I'll close this for now but please feel free to reopen if there are any questions or updates.

[zjorgensenbits]

Finally had a chance to upgrade and test this and it works great -- exactly what I was hoping for.
However, I believe I discovered a bug. If I click launch, everything works fine, but if I first click "show full config" and then click launch (even if I don't edit the config), the notebook immediately terminates. The log shows the following error: [2021-07-23, 17:13:24] test-notebook was terminated: container failed with non-zero exit code: error creating container: Error response from daemon: Duplicate mount point: /user_data/.

In our case, all of our bind mounts are defined in the master config. So it appears that if you hit show full config, it causes the bind mounts defined in the master config to be included again in the config file which causes this error.

[vishnu2kmohan]

Thanks for the report @zjorgensenbits, we'll take a look.

[hamidzr]

@zjorgensenbits do you mind expanding on the click launch you mention a bit?

If I click launch, everything works fine,

Is it that all the form inputs are empty like https://user-images.githubusercontent.com/12127420/122833216-6f12c200-d2a1-11eb-9e5c-ff9302874148.png or is it that some of those are changed, either at the time of launch or from a saved previous usage.

[zjorgensenbits]

I can select a template and resource pool and then click the Launch button at the bottom and everything works as expected. However, if instead of clicking the Launch button I instead click the "Show Full Config" button and then subsequently click the Launch button on that modal, the notebook fails to start.

[hamidzr]

that's helpful thank you. I suspect it'd fail with the same error if you leave the fields in the form empty and click launch. We'll look into this.

[zjorgensenbits]

Yes, I believe the first time I tried the new feature the fields were blank and hitting launch caused a similar failure. Now though, since I've already selected options it seems to remember those options in the future.

[hamidzr]

steps to reproduce: with the following in master config. a single bind mount should be enough as well

        task_container_defaults:
          bind_mounts:
            - host_path: /tmp/abc
              container_path: /tmp/c-abc
            - host_path: /tmp/xyz
              container_path: /tmp/c-xyz

the first request below fails with the reported error whereas the second one does not. we use the preview parameter to resolve the presented config in edit full config flow

curl-da.sh "${DET_MASTER}/api/v1/notebooks" \
  --data-raw '{"config":{"bind_mounts":[{"container_path":"/tmp/c-abc","host_path":"/tmp/abc","propagation":"rprivate","read_only":false},{"container_path":"/tmp/c-xyz","host_path":"/tmp/xyz","propagation":"rprivate","read_only":false}],"description":"Notebook (sincerely-unified-moray)","entrypoint":null,"environment":{"add_capabilities":null,"drop_capabilities":null,"environment_variables":{},"force_pull_image":false,"image":{"cpu":"determinedai/environments:py-3.7-pytorch-1.9-lightning-1.3-tf-2.4-cpu-2409e48","gpu":"determinedai/environments:cuda-11.1-pytorch-1.9-lightning-1.3-tf-2.4-gpu-2409e48"},"pod_spec":null,"ports":null},"resources":{"agent_label":"","devices":null,"resource_pool":"default","slots":1,"weight":1}}}'


curl-da.sh "${DET_MASTER}/api/v1/notebooks" \
  --data-raw '{"config":{"resources":{"slots":1}}}'

It looks like an issue with how we're merging the config posted through the request and existing defaults in the cluster when it's an array type