Override paketo-buildpacks download URIs about bellsoft-liberica

[bwgjoseph]

Hi @ekcasey, I hope it's ok to comment here since its closed (if opening a new issue is better, let me know).

I was hoping for a easier way to configure the URL, by hopefully through some kind of environment variable like how you can define for $BP_JVM_VERSION. So it could be something like $BP_JVM_URL.

I am working on an offline environment, hence, there is no way to hit github or any URL. We do have our own private repository that can host the files. The current way that I saw on SO seem to be a little too much work for defining URL. So I would really like to see if there is an easier way to do so.

Thanks!

[vnobo]

I have the same problem. It's a headache

[dmikusa]

1. Install https://github.com/dmikusa-pivotal/binding-tool, follow the instructions in the readme.
2. Run bt dm paketo-buildpacks/bellsoft-liberica. The tool will sit for a minute as it downloads the dependencies from the buildpack. After it's finished, it will write binding files in the current directory. Repeat for other Paketo buildpacks where you want to create dependency mappings.

The next part varies a bit based on the tooling you're using.

With pack, you need to run pack build <img> --volume $PWD/bindings:/bindings --env SERVICE_BINDING_ROOT=/bindings .... This will tell pack to include the bindings we just created. When the build runs, watch the output of Bellsoft Liberica, you should see it use file:// URLs to download the JDK/JRE. Those are the files bt dm downloaded.

With Spring Boot build tools, you need to add the volume mount and the env variable through your pom.xml or build.gradle file. Use the same values, but replace $PWD with the actual path to your current directory.

If you're using the creator directly, I believe but didn't try this, that you just need to set SERVICE_BINDING_ROOT to the path where your bindings exist. When you use the creator directly, it's your responsibility to set up the environment in which it's going to run so you need to get the binding files into the VM or container where you're running it, then set SERVICE_BINDING_ROOT to the location where you put them.

[vnobo]

so ,Now still can not do the custom JDK download path, offline can not be completed?

[GeTOUO]

我很难理解如此优秀的paketo-buildpacks项目，按理说对于用户应该是极大程度提升效率的。但是为什么偏偏在构建过程中，选择了每次都重新下载依赖包这么艰难的方式。哪怕给个配置，或者支持从本地读取下载好的文件也好啊

[nebhale]

There is not currently a way to change the download URIs to another arbitrary set without modifying the builder's filesystem with new URIs. I'm also not sure that there's a good, general purpose syntax for doing such a thing across the ~30 dependencies that are referenced via the builder, today.

Is the inability to access Github a blanket ban, or just that the traffic must go through a corporate proxy?

[nebhale]

We've got an idea of how we'll deliver this and put it on the backlog.

paketo-buildpacks/libpak#3

[dokaspar]

Hi @nebhale, the libpack#3 issue is already closed now... is it also released? And if so, what exactly would I have to do in order to override the download URL?

[nebhale]

@ekcasey is doing one final polishing round and then a round of releases at which point she'll update this issue. The functionality is implemented as a binding (or multiple bindings) with type of dependency-mapping. Bindings are effectively Kubernetes Secrets or ConfigMaps but can be implemented with pack and Docker just as easily. What you're looking for is a mapping from a SHA256 key, to a URI value. You'd expect to see something like this for the latest release of this buildpack:

├── 0ba6fa26b32e4b906ab460a7cdb70ebded95ea353fdda93bd7f5792300b9cd43 => <uri>
├── 25b312cfa20e19f09935e7dd57843bf3bf4e2013c63eea67b32846dfc742f80d => <uri>
├── 2e31cf143774a9b32f681dd34314202ac75aebe223297049c3ca55d0e1b70cbe => <uri>
├── 5592cf8d42130c711efca194b9a1acb54bf426bb9cbe21535905f03dff342c17 => <uri>
├── a3092627b082cb3cdbbe4b255d35687126aa604e6b613dcda33be9f7e1277162 => <uri>
├── b4cb31162ff6d7926dd09e21551fa745fa3ae1758c25148b48dadcf78ab0c24c => <uri>
├── decab4e27fadd5508cc9212ed959c3b8772a8c3b49f379f6ccfa1a9edfb179d8 => <uri>
├── efa6d87993ff21615e2d8fc0c98e07ff357fc9f3b9bd93c2cf58ba7f2b6fd2e0 => <uri>
└── type                                                             => `dependency-mappings`

If you're using pack you can pack build ... --volume <directory>:/bindings/dependency-mappings --env SERVICE_BINDING_ROOT=/platform/bindings to pass this directory in (once everything is complete).

[ekcasey]

This feature was release in version 2.11.1 with the UX described above by @nebhale

[dokaspar]

Hi @nebhale,

I'm not fully sure about this new feature, but it doesn't seem to be what I meant 😳 It sounds pretty complicated and inflexible to define a mapping as described in libpack#3:

[[mapping]]
id = "jdk"
version = "8.0.242"
uri = "https://internal:8080/jdk8.tar.gz
[[mapping]]
id = "jdk"
version = "11.0.6"
uri = "https://internal:8080/jdk11.tar.gz
[[mapping]]
id = "jre"
version = "8.0.242"
uri = "https://internal:8080/jre8.tar.gz
[[mapping]]
id = "jre"
version = "11.0.6"
uri = "https://internal:8080/jre11.tar.gz

Instead of repeating the exact same https://internal:8080 over and over again, wouldn't it be possible to set this via some base_uri property (which may default to github.com),

I envision that that the logic behind this statement...

Downloading from https://github.com/bell-sw/Liberica/releases/download/14.0.1+8/bellsoft-jre14.0.1+8-linux-amd64.tar.gz

... would become this:

Downloading from ${base_uri}/bell-sw/Liberica/releases/download/14.0.1+8/bellsoft-jre14.0.1+8-linux-amd64.tar.gz

Regards,
Dominik

[ekcasey]

@dokaspar Thanks for the feedback. I am open to iterating on the UX here, but I want to understand your use case better. Given your proposal I am curious if you have considered simply using an HTTPS proxy (https://paketo.io/docs/buildpacks/configuration/#proxy-configuration)?

[liaozan]

I have the same problem

[dmikusa]

I am working on an offline environment, hence, there is no way to hit github or any URL. We do have our own private repository that can host the files. The current way that I saw on SO seem to be a little too much work for defining URL. So I would really like to see if there is an easier way to do so.

It's a little cumbersome, but dependency-mapping is the only option at this time, short of packaging your own buildpack images and bundling assets in the images (we don't really recommend this though, as it's a non-trivial amount of work as well as an ongoing maintenance burden). Using a mapping should do what you want. ie. allow you to take an existing dependency in the buildpack and host that on an internal server.

If you don't want to manage the binding files directly, I have a little tool that you can run which will create the files given the required input. See https://github.com/dmikusa-pivotal/binding-tool.

On the long-term front, we're examining ways that we can better distribute assets with buildpacks. This would include having an easier way to redistribute assets in an offline environment. This isn't a quick change though because it depends on changes upstream in the buildpacks spec.

If you have further feedback or suggestions, I would request you open a new issue. It'll help us keep organized and make sure we don't miss anyone's feedback. Thanks.

[kiranpatel11]

same problem,

Can some one provide a working example for overriding the dependency-mapping bindings for BellSoft Liberica JRE. I also want to see how the config works with the /cnb/lifecycle/creator