science-computing / butido Goto Github PK

We might introduce "test" phases that need special handling from butido, for testing a successfully built and packaged software.

Discussion required.

It seems that because of the environment filtering, the "find-artifact" subcommand does not work properly.

Right now, dependencies are specified as a list of strings, where the strings are then parsed.

It might be beneficial, though, to specify them as key-value pairs, so they can be easily overwritten.

"Error: WouldCycle"

this error occurs if 2 packages depend on each other
location of at least one of the pkg.toml files that triggered the error would be helpful

This might hurt reusability if meta-information is interpolated into the packaging script (e.g. date), but maybe it is worth considering making such metadata available.

For example:

• Date
• "root" package of the build

more?

$ butido source download libxyz =1.2.3
[00:00:01] (100%): █████ | Repository loading finished
[00:00:00] (100%): █████ | Succeeded 0/0 downloads
$ echo $?
0

" Succeeded 0/0 downloads" should be an error and/or fail and tell the user about it

butido build --help
[...]
-L, --write-log
With this flag set, butido does not only write the build logs to database, but also to
the configured
log directory.

        The log of a build is written to `<log_dir>/<build id>.log`.
      [...]

logs are currently named $UUID.log
would be helpful to name them $UUID-$app-$version-$distroshortname.log

Putting an example repository in a github-actions workflow to test butido against it would be a nice idea.

It turned out that finding all packages that overwrite a specific phase would be beneficial for package maintainers.

A --dry-run flag could be added to the "build" subcommand to do everything except executing the build, so that one can investigate what would happen.

butido release only support adding and removing packages but the frontend currently does not yet provide a means to list all current releases for a store (or for all stores? ;) )

butido is version 0.3.0 atm, but with a lot of tests going on it could help to be able to output the commit id a butido binary was built on
maybe this is already possible and i just could not figure out how to get the commit id?

Because why not?

Issue to start discussion about more template helpers provided by butido.

Not sure whether we need any, but here's the place to discuss if.

what exactly is required to reuse an artifact?
when can a resuse happen and if not: what is stopping it?

When submitting a build, a comment could be added that is then stored in the database.

This would improve traceability of builds.

This is more of a perspectival issue.

So, right now the issue is that running butido from three different workstations from three different users results in a lot of builds on the build infrastructure. The individual instances of butido don't know from eachother and thus submit jobs to the builders as they see fit.

Having a central instance "butido-server" that schedules builds would enable better handling of load, but also some neat optimizations of build ordering.

Also, splitting a "server" part from the current CLI approach would reduce complexity in the CLI implementation, as the CLI would only be a thin-client (in the long run).

There are, of course, some subjects to think about:

The first step would be to make the server as slim as possible, and let the CLI compute everything. The server would only be responsibe for scheduling the individual jobs to the builder nodes.

Later on, though, one could think of a more advanced setup, where the server computes the scripts, the DAG and the optimal execution path. This is not trivial though.

Having a server component in the setup would also introduce other issues: build artifacts must then be stored on the server and synced to the client (the maintainer) on-demand. Definitively do-able.

The database would be another instance.
Caching of some form could be introduced (memcached, redis) to ease the load on the DB (build logs, etc are rather constant and do not change that often, so in-memory caching would be easy)

And while thinking about that, it might even be possible to split the server instance into some form of micro-services. One central scheduler instance and then "agents" which handle the build-orchestration of individual jobs on the builder instances. The builder instances could run (rather small) agents themselves, so that the communication between the orchestrating-parts and the executing-parts would be in our ecosystem.

After that, the executing-parts could implement different backends, not only docker. Building on non-x86_64 (think ARM!!!) could be rather easy to add!

The database does not yet contain build metrics, but tracking such metrics could improve the usability quite a bit.

E.G:

• Tracking the time a specific package build took could be used to forecast future build times (e.g. of package updates even)
• Success of builds is not yet tracked in the DB, but could significantly reduce load when searching for builds

E.G.:

• build package a
• build package b

both depend on c, but c gets rebuild when building b

What

• Bug
• Feature Request
• Question
• Other

There is https://github.com/lycheeverse/lychee which could be used to check whether all download links are valid, without re-downloading all the things!

• Drop the checklist from the PR template and check if the commits are signed off: #110
• Fix caching: #131
• Make the cargo-deny warnings more visible / fatal
• Avoid warnings (etc.)
• Stop ignoring cargo clippy warnings Lint with the beta toolchain as well (but only make the results with the MSRV binding): #111
• Analyze the security
• CLI/integration testing: #43
• Nix build (sandboxed and reproducible)
• CD? (upload binary / build artifact)
• Enforce consistent formatting (rustfmt) via CI: #263
• Other improvements (TODO)

error on other task stops all jobs, but which job triggered it?
with some patience and scrolling you can find out yourself, but explicitly name the package which had the error in the last line (small summary) would be nice

or just print it in red, so scrolling and finding it is no hassle

What

• Bug
• Feature Request
• Question
• Other

When loading the repository fails, the error message is not verbose enough (does not tell where the loading failed, for example).

What

• Bug
• Feature Request
• Question
• Other

This would be necessary to test new build endpoints in a large environment where one would need to build possibly several times to get butido to consider the new one.

load balancer does not balance within one submit
e.g. submit target with 300 dependencies -> almost all jobs run on the same endpoint / build host
next submit -> almost all jobs run on the next endpoint / build host

simple improvement of the current situation:
round-robin the jobs:
e.g. current job number modulo number of endpoints

related to #48

old:

[docker]
# Images which can be used to build
# images not listed here are automatically rejected
images = [
    "local:rh8-default",
    "local:rh8-rust-1.56-default",
    "local:sles15-go1.16.3-default",
]

new:

images = [
 { name = "local:rh8-default", shortname = "rh8" },
 { name = "local:rh8-rust-1.56-default", shortname = "rh8rust" },
 { name = "local:sles15-go1.16.3-default", shortname = "s15go" },
] 

discussion result:
this is a breaking change of the config format, so we bump butido version (to 0.4.0)

the progress bar during builds does use up more space than needed
half of it would be enough

goal:
quicker iterations while putting together a new build script

current situation:
if the top level pkg.toml was modified no reuse is possible for the next submit
this can be pretty annoying and time consuming if one or more dependencies take a long time to build

feature request:
if you know what you are doing (TM) you should be able to manually force a reuse of artifacts in a certain staging directory

The database can be slimmed a bit:

• ENV can be fetched from the git history of the package repository
• The script can be recomputed from the git history of the package repository
• The package can be fetched from the git history of the package repository

This information is stored in the DB right now, but does not have to be.

Backwards compatibility might be an issue, though.

Checking whether the package repository is clean could happen sooner as it does right now.

related to #89

add "Source Timestamp: VERGEN_GIT_COMMIT_TIMESTAMP" in output of "butido --version"

[00:00:00] (100%): █████ | Source verification failed
Error: Source missing: /scr/prokop/butido/sources/mpfr-4.1.1/src.source

Why not download the source on the fly then go on with the build?

$ butido db jobs -L 1
│ Submit │ Job │ Time │ Endpoint │ Success │ Package │ Version │
│ 40c0be74-0b77-457c-86a0-a1cd4f4cd1a9 │ ed855e87-0537-4b34-a45e-a7725e8b8e26 │ 2022-11-14 10:34:46 │ host1 │ yes │ demo-app │ 1.2.3│

butido db jobs does not list linux distro short name (e.g. "rh8", "sles15", ...)
location for the new column: far right (after "package" and "version")

After merging #41

$ butido build tree -I local:rh8-default -L 1.8.0
error: Invalid value 'false' for '--env': Error during validation: 'Incomplete' is not a key-value pair

Originally posted by @christophprokop in #41 (comment)

all issues related to "release workflow"

#142
#143

This would be a really good idea, because right now butido uses log, but in general the tracing ecosystem should be used for async code.

That change is rather simple IMO (but one learns to navigate the codebase rather well, as potentially every async function and every await should be annotated). May be perfect for someone learning the project codebase.

[00:00:40] (100%): █████ | [bonnie/14e6cba e9fd99d2-fdf0-4cfa-9c8a-8cd6cd86dd05 libgl2ps 1.4.2]: finished successfully                                                                                           Packages created:
-> /scr/stage/b5e9e6f6-1e55-427e-8ea6-d503c7c99f7f/opensoftware_libgl2ps_1.4.2-1.4.2-1.el7.x86_64.rpm

remove "->" at the beginning of the line, so copying is more convenient :)

We might think about whether a package without any sources can exist.

E.G. meta-packages, that only pull in other packages.

We get that information from GIT_AUTHOR_NAME, it should be added in the database so that this information is traceable.

When building a target with a lot of dependencies (so not all lines are visible on a single screen any more), it would be helpful to have the progress line of the main target at the bottom (at not on top):

[00:06:30] ( 0%): ░░░░░ | [51357591-4a9e-4596-9cd8-d2ad592aea52 example 1.2.3]: Waiting (7/285)...

this is not critical but rather nice to have :)

Error: Loading the repository

Caused by:
    missing field `name`

This error message occurs if any subtree of pkg.toml files is broken
would be really helpful to know where exactly the problem occurs (files? subtree?)

$ butido db jobs -L 1
│ Submit │ Job │ Time │ Endpoint │ Success │ Package │ Version │
│ 40c0be74-0b77-457c-86a0-a1cd4f4cd1a9 │ ed855e87-0537-4b34-a45e-a7725e8b8e26 │ 2022-11-14 10:34:46 │ host1 │ yes │ demo-app │ 1.2.3│

shorten header fields:
s/Success/OK?/
and
s/Endpoint/Host/

There are some flags that could be added to the "db releases" subcommand:

• "--sort=date"
• "--sort=name"

more?

Error: build command failed

[...]

       butido db log-of 038255b5-ac92-4f73-8880-44a7f0870abf | less -SR +G

1: Finalizing container
2: Copying the TAR stream to the staging store
3: Concatenating the output bytestream
4: Copying item from container 30a7db8788fc1946503cc736b3a627a1fe80682c04dc75f6b5b3c67ccd521c84 to host
5: Docker Error: 404 Not Found

----------------------------

this error is a bit misleading
the logfile shows the real problem:

/script: line 730: CONFIGURE_FLAGS[@]: unbound variable

(Ensure the path is checked into the package repository though)