foundriesio / ci-scripts Goto Github PK
View Code? Open in Web Editor NEWScripts used in Foundries.io CI Projects
License: BSD 3-Clause "New" or "Revised" License
Scripts used in Foundries.io CI Projects
License: BSD 3-Clause "New" or "Revised" License
That looks nice to try and hide things. But it winds up producing a message like this:
ERROR:System Image Assembler:Failed to assemble a system image(s): 'image-file'
I'm assuming there was some KeyError thrown somewhere. But lacking the exception (and really stack trace) makes things like this hard to dig into.
The Foundries Factory currently has rudimentary support for managing tags that are included with OTA Targets as the result of a CI build. I'm opening this issue to explain the current state of tagging, what is missing, and then get an agreement on the solution.
Platform Build - A build created by a change to the LMP/OE. This is the base OS image
Container Build - A build created by a change to containers.get.
Target - This a TUF entry in a factory's targets.json file. It represents what we should think of as an immutable combination of the Platform Build's OSTree hash + the output of a Container Build.
Tag - A Target has a "custom" section where we maintain a list of Tags. The tags can be used to say things like "this is a development build", or this is a "production" build.
Currently our tagging logic assumes one Platform branch and one Containers branch are paired together. The way this works is as follows:
A change to the "devel" branch on containers.git would cause new docker containers to get built. It then creates a new Target with these containers referenced in the docker-apps section. A Target needs a Platform build to be complete, so it scans through targets.json to find the previous Target with the "devel" tag. It then copies its OSTree hash to produce the new Target.
A change to the "devel" Platform is similar. It produces a new Target with the OStree hash from its build. It then finds the previous "devel" Target in targets.json and copies the docker apps that are defined in it.
This is al done by having a single environment variable defined in CI: OTA_LITE_TAG
.
There are two new scenarios this logic can't cover:
We want to add a new "postmerge-stable" Platform build. However, we just want to use the current containers from the most recent "postmerge" merge.
This also means that a change to the Containers project would now need to create 2 Targets - a "postmerge" and "postmerge-stable".
There is a 3rd scenario users have: There are 2 Container tags that should be based on the same Platform. This turns out to really be the same as the previous two scenarios.
OTA_LITE_TAG
isn't expressive enough. You actually need to express:
I think we could (ab)use the currrent OTA_LITE_TAG to express this. The format could be defined like:
TAG[:INHERIT_TAG][,TAG[:INHERIT_TAG]....]
postmerge-stable Platform build should grab latest postmerge Containers:
OTA_LITE_TAG: postmerge-stable:postmerge
Which means a change to containers.git produces 2 tags now (postmerge and postmerge-stable):
OTA_LITE_TAG: postmerge,postmerge-stable
One platform with multiple containers. Let's call things:
So a Platform devel-X change could be expressed with:
OTA_LITE_TAG:devel-X,devel-Y
A change to either container stays "normal". ie its just OTA_LITE_TAG: devel-Y
Line 212 in cb79078
It looks like the EXTRA_TAGS option may have been broken with the move to docker-buildx.
We need to add a generic way to pass docker --build-arg(s) to be used in the Dockerfile. To make things more clear I think build.sh should check for environment variables starting with DOCKER_ARG_
and pass them all as --build-arg
arguments. eg:
DOCKER_ARG_FOO=foo
DOCKER_ARG_BAR=bar
Would add to our docker build command: --build-arg FOO=foo --build-arg BAR=bar
It will then be up to a factory's container.yml to pass the values they want into the CI job.
There are some variables that are important and should not have to be set via lmp.params
, such as IMAGES
and DOCKER_SECRETS
. For example, if we do not define IMAGES
then it will produce an error. My understanding is that lmp.params
is intended to be a user-configured section, with only user additions. So it makes sense to separate these two, as currently we set:
lmp:
params:
IMAGE: lmp-factory-image
which encroaches upon the user's space. The same will be true if we introduce other variables here by default in future, instead of creating new directives for these variables.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.