osbuild / weldr-client Goto Github PK

Go client library and cmdline tool for WELDR API servers like lorax-composer and osbuild-composer

License: Apache License 2.0

Makefile 0.81% Go 99.10% Shell 0.09%

weldr-client's Issues

`make rpm` on git checkout fails on GPG signing of the archive

I wanted to test a version of weldr-client with the support for cross-distro building and while trying to build RPMs from the git checkout, I got an error due to failed signing:

$ make rpm
sed -e "s/%%VERSION%%/35.1/g" -e "s/%%GPGKEY%%/908AF740C0F898A1079968CDC5887AD51D9F3C2D/g" < weldr-client.spec.in > weldr-client.spec
make -s changelog >> weldr-client.spec
mkdir -p /home/thozza/devel/weldr-client/rpmbuild/SPECS
cp weldr-client.spec /home/thozza/devel/weldr-client/rpmbuild/SPECS
git archive --prefix=weldr-client-35.1/ --format=tar.gz HEAD > weldr-client-35.1.tar.gz
gpg --armor --detach-sign -u 908AF740C0F898A1079968CDC5887AD51D9F3C2D weldr-client-35.1.tar.gz
gpg: skipped "908AF740C0F898A1079968CDC5887AD51D9F3C2D": No secret key
gpg: signing failed: No secret key
make: *** [Makefile:53: sign] Error 2

Please consider not signing the archive as part of rpm target.

composer-cli compose log says compose is still running.

According to the command help, compose log is meant to get information of the running build, however, it complains the compose is still running, which sounds like a contradiction.

[root@fedoravm image-builder]# composer-cli compose list
ID                                     Status    Blueprint   Version   Type
a4abfd36-0b79-4ccb-a6ac-3edea46ec575   RUNNING   fedora-nx   0.0.1     image-installer
17bda711-098a-413d-bbd8-c4f6f7ac6c34   FAILED    fedora-nx   0.0.2     image-installer
b2f165f0-cb6d-4803-880d-1d62b365f6dd   FAILED    fedora-nx   0.0.0     image-installer
[root@fedoravm image-builder]# composer-cli compose log a4abfd36-0b79-4ccb-a6ac-3edea46ec575 100
Build a4abfd36-0b79-4ccb-a6ac-3edea46ec575 is still running.

Migrate from weldr-api to cloud-api.

weldr-client currently speaks to the weldr-api as provided by osbuild-composer. In the longer term I'd like to move this over to the cloud-api, also provided by osbuild-composer.

Most important is how we deal with blueprints. These are currently being parsed on the weldr-api side of things and with this change they would need to be parsed and converted into a cloud-api type request on the client side.

We would likely split out the parsing and validation of blueprints to a separate library as the service has also indicated interest in copy/pasting in a blueprint.

composer-cli blueprints save produces incorrect blueprints

Describe the bug
composer-cli blueprints save creates a blueprint .toml where the user's UID and GID are floating point numbers, e.g. a UID of 1200 will be 1200.0 in the resulting .toml

Environment

OS version (/etc/os-release and /etc/redhat-release): Fedora 35
osbuild-composer version (rpm -qi osbuild-composer)

Name        : osbuild-composer
Version     : 42
Release     : 1.fc35
Architecture: x86_64
Install Date: Tue 01 Feb 2022 02:08:27 PM CET
Group       : Unspecified
Size        : 17577
License     : ASL 2.0
Signature   : RSA/SHA256, Wed 12 Jan 2022 02:59:40 PM CET, Key ID db4639719867c58f
Source RPM  : osbuild-composer-42-1.fc35.src.rpm
Build Date  : Wed 12 Jan 2022 02:53:03 PM CET
Build Host  : buildvm-x86-17.iad2.fedoraproject.org
Packager    : Fedora Project
Vendor      : Fedora Project
URL         : https://github.com/osbuild/osbuild-composer
Bug URL     : https://bugz.fedoraproject.org/osbuild-composer
Summary     : An image building service based on osbuild
Description :

A service for building customized OS artifacts, such as VM images and OSTree
commits, that uses osbuild under the hood. Besides building images for local
usage, it can also upload images directly to cloud.

It is compatible with composer-cli and cockpit-composer clients.

To Reproduce
Steps to reproduce the behavior:

Create an original_blueprint.toml file with these contents:

name = "BPSaveBug"
description = ""
version = "0.0.1"

[[customizations.user]]
name = "admin"
password = "$6$qL0yRFzeHbv5xvh9$DLgYdwRL5dYj0S9R941AWbjA6lN/ecJr/htnyPvct93tke2sp81qy9F7EHAZmIRMQtlCUT82Q2DYaoAqRJ3CG."
groups = ["wheel"]
uid = 1200
gid = 1200

Push and save the blueprint:
composer-cli blueprints push originalblueprint.toml
composer-cli blueprints save originalblueprint

The resulting BPSaveBug.toml has incorrect floating point values for gid and uid:

description = ""
distro = ""
groups = []
modules = []
name = "BPSaveBug"
packages = []
version = "0.0.1"

[customizations]

  [[customizations.user]]
    gid = 1200.0
    groups = ["wheel"]
    name = "admin"
    password = "$6$qL0yRFzeHbv5xvh9$DLgYdwRL5dYj0S9R941AWbjA6lN/ecJr/htnyPvct93tke2sp81qy9F7EHAZmIRMQtlCUT82Q2DYaoAqRJ3CG."
    uid = 1200.0

Expected behavior

description = ""
distro = ""
groups = []
modules = []
name = "BPSaveBug"
packages = []
version = "0.0.1"

[customizations]

  [[customizations.user]]
    gid = 1200
    groups = ["wheel"]
    name = "admin"
    password = "$6$qL0yRFzeHbv5xvh9$DLgYdwRL5dYj0S9R941AWbjA6lN/ecJr/htnyPvct93tke2sp81qy9F7EHAZmIRMQtlCUT82Q2DYaoAqRJ3CG."
    uid = 1200

Additional context
Add any other context about the problem here.

Using weldr API client for go binding

Hello,

Is there any plans on structuring this so that other Go applications can
use the weldr client that this has?

I don't see a good way to import it right now so I think we'll have to copy the code.

Best regards

JSON Output for multiple requests isn't a JSON list

The JSON output for commands like list that send more than one request is not a JSON list, it is a series of distinct objects. It would be nice if this could be JSON compliant so it can be passed directly into a parser.

Cannot upgrade from composer-cli on Fedora 34

$ rpm -q composer-cli
composer-cli-34.9-5.fc34.x86_64
$ sudo dnf install --enablerepo=updates-testing weldr-client
Last metadata expiration check: 0:23:11 ago on Čt 22. dubna 2021, 15:05:27.
Dependencies resolved.
===============================================================================================================================================================
 Package                                Architecture                     Version                               Repository                                 Size
===============================================================================================================================================================
Installing:
 weldr-client                           x86_64                           35.1-1.fc34                           updates-testing                           2.6 M

Transaction Summary
===============================================================================================================================================================
Install  1 Package

Total size: 2.6 M
Installed size: 8.6 M
Is this ok [y/N]: y
Downloading Packages:
[SKIPPED] weldr-client-35.1-1.fc34.x86_64.rpm: Already downloaded                                                                                             
Running transaction check
Transaction check succeeded.
Running transaction test
The downloaded packages were saved in cache until the next successful transaction.
You can remove cached packages by executing 'dnf clean packages'.
Error: Transaction test error:
  file /usr/bin/composer-cli from install of weldr-client-35.1-1.fc34.x86_64 conflicts with file from package composer-cli-34.9-5.fc34.x86_64
  file /usr/share/man/man1/composer-cli.1.gz from install of weldr-client-35.1-1.fc34.x86_64 conflicts with file from package composer-cli-34.9-5.fc34.x86_64

I think that maybe Obsoletes: composer-cli < 35.0 might solve it, I didn't test it though. We might also want Conflicts? Not sure...

Write release documentation

Document how to make a new upstream release.

`compose image` shouldn't use /tmp for temporary files

Currently, composer-cli compose image firstly downloads the image into /tmp. This causes the osbuild-composer's test suite to fail because on AWS, /tmp is on tmpfs and we don't have enough RAM to store the whole image. My proposal is to drop the temporary file and save the image directly in the current working directory.

use different toml library then BurntSushi/toml

BurntSushi/toml is unmaintained: https://github.com/BurntSushi/toml

We should probably switch to using something else sooner rather than later.

Add option to download created image to a specific file name

As the subject says, currently composer-cli compose image xxx downloads to a pre-defined image name (based on the UUID) but this isn't optimal for automation, as it forces one to parse the output and rename the image in a 2nd step and, because Ansible doesn't support moving/renaming of files, it requires a copy and/or using a command to move it. Anyway, it's less than nice and being able to give directly the name of the downloaded image, would make it easier.

No cli feedback on 'sudo composer-cli compose results'

When calling sudo composer-cli compose results I get not feedback on stdout (as opposed to the docs which suggest the name and size of the result should be printed).

I know this is a minor / cosmetic issue but at first I thought something had gone wrong until I figured it out.
(Using F34 and osbuild from the repos.)

Add support for upload status

The compose info response from the server can also contain the status of uploads, these should also be printed.
The trick is to figure out how to include them in the current output without making it messy.

Fix Cobra flag usage during testing

As pointed out in pr #7 the flags don't get reset between tests because the Cobra struct persists for all the tests.
Need to figure out a nice way to reset things so that you don't need to clear them all at the start of every test.

composer-cli --json blueprint changes returns multiple json objects

the old composer-cli:

$ sudo composer-cli --json blueprints changes empty
{
    "blueprints": [
        {
            "changes": [
                {
                    "commit": "14cf307a846a66b4ae184aa79b00ffb5d7de2cb1",
                    "message": "Recipe empty, version  saved.",
                    "revision": null,
                    "timestamp": "2021-04-22T15:59:36Z"
                },
...
            ],
            "name": "empty",
            "total": 15
        }
    ],
    "errors": [],
    "limit": 15,
    "offset": 0
}

the new weldr-client:

$ sudo composer-cli --json blueprints changes empty
{
    "method": "GET",
    "path": "/blueprints/changes/empty?limit=0",
    "status": 200,
    "body": {
        "blueprints": [
            {
                "changes": [
                    {
                        "commit": "14cf307a846a66b4ae184aa79b00ffb5d7de2cb1",
                        "message": "Recipe empty, version  saved.",
                        "revision": null,
                        "timestamp": "2021-04-22T15:59:36Z"
                    },
...
        ],
        "errors": [],
        "limit": 0,
        "offset": 0
    }
}
{
    "method": "GET",
    "path": "/blueprints/changes/empty?limit=15",
    "status": 200,
    "body": {
        "blueprints": [
            {
                "changes": [
                    {
                        "commit": "14cf307a846a66b4ae184aa79b00ffb5d7de2cb1",
                        "message": "Recipe empty, version  saved.",
                        "revision": null,
                        "timestamp": "2021-04-22T15:59:36Z"
                    },
...
                ],
                "name": "empty",
                "total": 15
            }
        ],
        "errors": [],
        "limit": 15,
        "offset": 0
    }
}

I'm not sure what's the expected behavior here. I would say that a user expects a valid JSON when passing the --json flag. On the hand, the output is actually a valid JSON stream. Not sure, what do you think @bcl?

[root@kvm-08-guest31 osbuild-composer]# composer-cli compose start whatever qcow2
ERROR: UnknownBlueprint: Unknown blueprint name: whatever
[root@kvm-08-guest31 osbuild-composer]# echo $?
1
[root@kvm-08-guest31 osbuild-composer]# composer-cli --json compose start whatever qcow2
{
    "method": "POST",
    "path": "/api/v1/compose",
    "status": 400,
    "body": {
        "errors": [
            {
                "id": "UnknownBlueprint",
                "msg": "Unknown blueprint name: whatever"
            }
        ],
        "status": false
    }
}
[root@kvm-08-guest31 osbuild-composer]# echo $?
0

Ran using weldr-client-35.1-4.el9.x86_64

invalid-url URL `%{gourl}`

user@desktopa weldr-client € make scratch-srpm
sed -e "s/%%VERSION%%/35.6/g" -e "s/%%GPGKEY%%//g" < weldr-client.spec.in > weldr-client.spec
make -s changelog >> weldr-client.spec
mkdir -p /home/supakeen/dev/src/work/weldr-client/rpmbuild/SPECS
cp weldr-client.spec /home/supakeen/dev/src/work/weldr-client/rpmbuild/SPECS
git archive --prefix=weldr-client-35.6/ --format=tar.gz HEAD > weldr-client-35.6.tar.gz
mkdir -p /home/supakeen/dev/src/work/weldr-client/rpmbuild/SOURCES
cp weldr-client-35.6.tar.gz* rpmbuild/SOURCES/
rpmbuild -bs \
	--define "_topdir /home/supakeen/dev/src/work/weldr-client/rpmbuild" \
	--define "commit 35.6" \
	--with tests \
	--without signed \
	rpmbuild/SPECS/weldr-client.spec
setting SOURCE_DATE_EPOCH=1661472000
Wrote: /home/supakeen/dev/src/work/weldr-client/rpmbuild/SRPMS/weldr-client-35.6-1.fc36.src.rpm
user@desktopa weldr-client € rpmlint rpmbuild/SRPMS/weldr-client-35.6-1.fc36.src.rpm
================================================================================================================================== rpmlint session starts ==================================================================================================================================
rpmlint: 2.2.0
configuration:
    /usr/lib/python3.10/site-packages/rpmlint/configdefaults.toml
    /etc/xdg/rpmlint/fedora.toml
    /etc/xdg/rpmlint/licenses.toml
    /etc/xdg/rpmlint/scoring.toml
    /etc/xdg/rpmlint/users-groups.toml
    /etc/xdg/rpmlint/warn-on-functions.toml
checks: 32, packages: 1

weldr-client.src: W: unexpanded-macro URL %{gourl}
weldr-client.spec:30: W: macro-in-comment %check
weldr-client.spec:168: W: macro-in-%changelog %gobuild
weldr-client.src: W: invalid-url URL %{gourl}

Add 'compose wait UUID' command to wait for a compose to finish/fail

Working on this in #125

osbuild / weldr-client Goto Github PK

weldr-client's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs