bootstrap-conda.sh failing on build (no .bashrc found) about initialization-actions

[nehalecky]

For reference, logs from master node:

• dataproc-initialization-script-0_output
• dataproc-startup-script_output

[nehalecky]

Was able to resolve simply by removing the sourcing calls to /root/.bashrc (no longer needed due to setting PATH env vars and other configs in global profiles). This properly allowed conda to be installed along with all dependencies, and also ensures updated path (exposing miniconda) for both root and non-root users. Will submit patch shortly.

[nehalecky]

OK, the PR above should resolve this. It has in my testing, including functionality contained in existing PRs depending on conda. Give it a try and we should get this merged asap. Thanks!

[dennishuo]

Sorry for the breakage! We indeed rolled out some minor cleanup of the VM environment for newly created clusters, and didn't intend to change anything in a breaking way. We haven't quite standardized the testing of known initialization actions in the release process yet, but in the future we should be able to catch these before any public releases. I'll take a look at the PR.

[nehalecky]

@dennishuo, thanks for the quick resolution. We look forward to contributing to the testing of init actions, and I believe a first step would be to expose some google compute engine resources to be for spinning up a minimal dataproc cluster and running a test suite. Internally, we will be implementing this soon, but believe it would be awesome if Google would contribute some resources to do the same.

Also, from our side, is there anything we can do to prevent such breakages in the future (any way to reference specific subminor dataproc versions)?

Thanks again!

[dennishuo]

@nehalecky We're definitely hoping to provide a nice sandboxed CI/test setup for running minimal test suites, unfortunately can't guarantee a timeline at this point since we're still shuffling around various prioritizations.

It's indeed possible to reference specific subminor versions, but the long-term deprecation policies cover major.minor version "tracks" and not specific subminor versions, so they may be subject to shorter lifecycles of being officially supported compared to the lifecycles of major.minor tracks.

That said, in this case the previous subminor version that we'd expect to still work with the old conda init action would be version "1.0.0", and the current version is "1.0.1". Pinning to 1.0.1 now would ensure workloads are unaffected when 1.0.2 comes out, but it'd be strongly recommended to switch to newer subminor versions as soon as possible; previous subminor versions wouldn't get bug fixes or new features.

We'll probably add a nicer way to query for the list of supported and deprecated subminor versions, but in the meantime you can at least see the complete list by creating a cluster and specifying a bogus version like --image-version=notavalidversion

[nehalecky]

@dennishuo, thank you for the comprehensive reply. We're excited about these future developments and will continue to work alongside you all to see this advanced! Have a great weekend!