sepulworld / aptly_cli

Aptly client

Hi there 👋

aptly_cli's Issues

Return code when Error occurs

Hi,

Is it possible to get an exit code different than 0 when aplty_cli command failing?

Cheers,

Vasilis

Improve CLI response format

Presently, CLI returns JSON received from Aptly API. It might be good to consider retuning something a bit more user friendly to read on command line.

$ aptly-cli repo_upload --dir test1 --name ssr-precise -trace
{"FailedFiles"=>[], "Report"=>{"Warnings"=>[], "Added"=>["mariadb-common_5.5.32+maria-1~precise_all added"], "Removed"=>[]}}

would look like...

$ aptly-cli repo_upload --dir test1 --name ssr-precise -trace
Added: mariadb-common_5.5.32+maria-1~precise_all

or if it failed to add...

$ aptly-cli repo_upload --dir test1 --name ssr-precise -trace
FailedFiles: mariadb-common_5.5.32+maria-1~precise_all

Basically,
puts ["FailedFiles"] if not empty.
puts ["Report"]["Warnings"] if not empty
puts ["Report"]["Added"] if not empty
puts ["Report"]["Removed"] if not empty

gpg_batch not returning a Boolean

$ aptly-cli publish_update --forceoverwrite --gpg_batch --gpg_passphrase 'TEST' --distribution test-platform
{"Batch"=>"true", "Passphrase"=>"TEST"}

Batch should not have a string as a value. It needs to be a Boolean

{"error"=>"json: cannot unmarshal string into Go value of type bool", "meta"=>"Operation aborted"}

Default /etc/aptly-cli.conf is not secure

It would leak password.

I suggest that aptly-cli tries to load:

1. ~/.config/aptly-cli/aptly-cli.conf
2. /etc/apt-cli/aptly-cli.conf

failedFiles vs FailedFiles

I'm seeing:

/opt/ruby/2.1.2/lib/ruby/gems/2.1.0/gems/aptly_cli-0.1.6/lib/aptly_repo.rb:123:in 'repo_upload': undefined method 'empty?' for nil:NilClass (NoMethodError)

in aptly_repo.rb; code references json_response["failedFiles"], but I see FailedFiles if I run the API manually.

Move from httmultiparty to httparty

Move from httmultiparty to httparty

httparty now supports multipart file uploads.

aptly-cli snapshot_list --all

Bug with aptly-cli snapshot_list

$ aptly-cli snapshot_list --sort all --trace
/Users/zanew/.rvm/gems/ruby-2.2.1/gems/aptly_cli-0.1.8/lib/aptly_snapshot.rb:25:in `snapshot_list': wrong number of arguments (2 for 0..1) (ArgumentError)
    from /Users/zanew/.rvm/gems/ruby-2.2.1/gems/aptly_cli-0.1.8/bin/aptly-cli:314:in `block (2 levels) in <top (required)>'
    from /Users/zanew/.rvm/gems/ruby-2.2.1/gems/commander-4.3.5/lib/commander/command.rb:178:in `call'
    from /Users/zanew/.rvm/gems/ruby-2.2.1/gems/commander-4.3.5/lib/commander/command.rb:178:in `call'
    from /Users/zanew/.rvm/gems/ruby-2.2.1/gems/commander-4.3.5/lib/commander/command.rb:153:in `run'
    from /Users/zanew/.rvm/gems/ruby-2.2.1/gems/commander-4.3.5/lib/commander/runner.rb:428:in `run_active_command'
    from /Users/zanew/.rvm/gems/ruby-2.2.1/gems/commander-4.3.5/lib/commander/runner.rb:68:in `run!'
    from /Users/zanew/.rvm/gems/ruby-2.2.1/gems/commander-4.3.5/lib/commander/delegates.rb:15:in `run!'
    from /Users/zanew/.rvm/gems/ruby-2.2.1/gems/commander-4.3.5/lib/commander/import.rb:5:in `block in <top (required)>'

foreceoverwrite causing error when using publish_update option

$ aptly-cli publish_update --distribution precise --forceoverwrite true
{"error"=>"json: cannot unmarshal string into Go value of type bool", "meta"=>"Operation aborted"}

test/Dockerfile no longer builds

$ docker build -t sepulworld/aptly_api_test ./test/
Sending build context to Docker daemon 75.26 kB
Step 1 : FROM debian:jessie
 ---> 1b088884749b
Step 2 : EXPOSE 8080
 ---> Using cache
 ---> 059055839ed0
Step 3 : RUN echo "deb http://repo.aptly.info/ squeeze main" > /etc/apt/sources.list.d/aptly.list; apt-key adv --keyserver keys.gnupg.net --recv-keys 2A194991; apt-get update; apt-get install aptly curl xz-utils bzip2 gnupg wget graphviz -y --force-yes; wget --quiet http://mirror.as24220.net/pub/ubuntu-archive/pool/main/z/zeitgeist/zeitgeist_0.9.0-1_all.deb -O /tmp/zeitgeist_0.9.0-1_all.deb; wget --quiet http://mirror.as24220.net/pub/ubuntu-archive/pool/main/z/zsh/zsh_5.1.1-1ubuntu1_i386.deb -O /tmp/zsh_5.1.1-1ubuntu1_i386.deb
 ---> Running in 000ec37cb330
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --homedir /tmp/tmp.OpKMIiSwnv --no-auto-check-trustdb --trust-model always --primary-keyring /etc/apt/trusted.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-jessie-automatic.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-jessie-security-automatic.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-jessie-stable.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-squeeze-automatic.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-squeeze-stable.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-wheezy-automatic.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-wheezy-stable.gpg --keyserver keys.gnupg.net --recv-keys 2A194991
gpg: requesting key 2A194991 from hkp server keys.gnupg.net
?: keys.gnupg.net: Host not found
gpgkeys: HTTP fetch error 7: couldn't connect: Connection timed out
gpg: no valid OpenPGP data found.
gpg: Total number processed: 0
Err http://httpredir.debian.org jessie InRelease

Err http://repo.aptly.info squeeze InRelease

Err http://httpredir.debian.org jessie-updates InRelease

Err http://security.debian.org jessie/updates InRelease

Err http://httpredir.debian.org jessie Release.gpg
  Could not resolve 'httpredir.debian.org'
Err http://security.debian.org jessie/updates Release.gpg
  Could not resolve 'security.debian.org'
Err http://repo.aptly.info squeeze Release.gpg
  Could not resolve 'repo.aptly.info'
Err http://httpredir.debian.org jessie-updates Release.gpg
  Could not resolve 'httpredir.debian.org'
Reading package lists...
W: Failed to fetch http://httpredir.debian.org/debian/dists/jessie/InRelease

W: Failed to fetch http://httpredir.debian.org/debian/dists/jessie-updates/InRelease

W: Failed to fetch http://security.debian.org/dists/jessie/updates/InRelease

W: Failed to fetch http://repo.aptly.info/dists/squeeze/InRelease


�
W: Failed to fetch http://httpredir.debian.org/debian/dists/jessie/Release.gpg  Could not resolve 'httpredir.debian.org'

W: Failed to fetch http://repo.aptly.info/dists/squeeze/Release.gpg  Could not resolve 'repo.aptly.info'

W: Failed to fetch http://httpredir.debian.org/debian/dists/jessie-updates/Release.gpg  Could not resolve 'httpredir.debian.org'

W: Failed to fetch http://security.debian.org/dists/jessie/updates/Release.gpg  Could not resolve 'security.debian.org'

W: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists...
Building dependency tree...
Package bzip2 is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

Package xz-utils is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Unable to locate package aptly
E: Unable to locate package curl
E: Package 'xz-utils' has no installation candidate
E: Package 'bzip2' has no installation candidate
E: Unable to locate package wget
E: Unable to locate package graphviz
/bin/sh: 1: wget: not found
/bin/sh: 1: wget: not found
The command '/bin/sh -c echo "deb http://repo.aptly.info/ squeeze main" > /etc/apt/sources.list.d/aptly.list; apt-key adv --keyserver keys.gnupg.net --recv-keys 2A194991; apt-get update; apt-get install aptly curl xz-utils bzip2 gnupg wget graphviz -y --force-yes; wget --quiet http://mirror.as24220.net/pub/ubuntu-archive/pool/main/z/zeitgeist/zeitgeist_0.9.0-1_all.deb -O /tmp/zeitgeist_0.9.0-1_all.deb; wget --quiet http://mirror.as24220.net/pub/ubuntu-archive/pool/main/z/zsh/zsh_5.1.1-1ubuntu1_i386.deb -O /tmp/zsh_5.1.1-1ubuntu1_i386.deb' returned a non-zero code: 127

file_upload fails with aptly-cli, but works with curl

I'm having trouble getting file_upload to work, and I can't find any obvious errors anywhere. Suspecting some kind of bug. Uploading a file by accessing the aptly API using curl works. Other aptly-cli commands work, such as file_list, repo_upload and publish_update.

My aptly-cli server is Debian 9.6, Ruby 2.3.3, aptly_cli 0.4.1.
My aptly api server is Debian 9.6, aptly 1.3.0.

File to upload:

# ls -l aptly-test-file.txt 
-rw-r--r-- 1 root root 22 Jan  8 09:33 aptly-test-file.txt

# cat aptly-test-file.txt
1234567890
abcdefghij

Uploading the file using aptly-cli causes the directory to be created on the server, but the file is not uploaded. The response also does not contain the filename as expected:

# aptly-cli file_upload --trace --upload aptly-test-file.txt --directory /testfile
opening connection to my.aptly.server:443...
opened
starting SSL for my.aptly.server:443...
SSL established
<- "POST /api/files/testfile HTTP/1.1\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nAccept: */*\r\nUser-Agent: Ruby\r\nContent-Type: multipart/form-data; boundary=-----------RubyMultipartPost\r\nContent-Length: 447\r\nAuthorization: Basic <redacted>\r\nConnection: close\r\nHost: my.aptly.server\r\n\r\n"
-> "HTTP/1.1 200 OK\r\n"
-> "Server: nginx/1.14.1\r\n"
-> "Date: Tue, 08 Jan 2019 08:42:46 GMT\r\n"
-> "Content-Type: application/json; charset=utf-8\r\n"
-> "Content-Length: 2\r\n"
-> "Connection: close\r\n"
-> "Strict-Transport-Security: max-age=15768000\r\n"
-> "\r\n"
reading 2 bytes...
-> "[]"
read 2 bytes
Conn close

My /etc/aptly-cli.conf looks like this:

# cat /etc/aptly-cli.conf 
---
:proto: https
:server: my.aptly.server
:port: 443
:username: aptlyuser
:password: <redacted>
:debug: true

Using curl to upload the same file works, and shows a significant difference in the reported Content-Length of the upload. I'm not sure exactly how this is calculated:

# curl -v -F [email protected] https://aptlyuser:<redacted>@my.aptly.server/api/files/testfile
*   Trying ip.of.my.aptly.server...
* TCP_NODELAY set
* Connected to my.aptly.server (ip.of.my.aptly.server) port 443 (#0)
[..handshake lines removed..]
> POST /api/files/testfile HTTP/1.1
> Host: my.aptly.server
> Authorization: Basic <redacted>
> User-Agent: curl/7.52.1
> Accept: */*
> Content-Length: 219
> Content-Type: multipart/form-data; boundary=------------------------61a6064b8ba6fb9f
> 
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200 
< server: nginx/1.14.1
< date: Tue, 08 Jan 2019 08:46:08 GMT
< content-type: application/json; charset=utf-8
< content-length: 32
< strict-transport-security: max-age=15768000
<
* Curl_http_done: called premature == 0
* Connection #0 to host my.aptly.server left intact
["testfile/aptly-test-file.txt"]

These requests are, as you can see, all done through Nginx, but I have tried to also talk directly to the aptly API on port 8080, which gives me the exact same results.

For completeness, this is the relevant part in the Nginx config that handles requests to /api/:

location /api/ {
  client_max_body_size 100M;
  auth_basic "Restricted";
  auth_basic_user_file /etc/nginx/.htpasswd.aptly;
  proxy_redirect	off;
  proxy_pass	http://localhost:8080/api/;
  proxy_redirect	http://localhost:8080/api/ /api;
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $remote_addr;
  proxy_set_header X-Forwarded-Proto $scheme;
  proxy_set_header Host $http_host;
  proxy_set_header Origin "";
}

And this is the supervisor config, used to start the aptly api serve:

[program:aptly]
command=/usr/bin/aptly api serve -no-lock
directory=/home/aptly
user=aptly
stdout_logfile=/var/log/supervisor/aptly-stdout.log
stderr_logfile=/var/log/supervisor/aptly-stderr.log

Add support for Package Reference API feature

From aptly.info api docs...

CREATE SNAPSHOT FROM PACKAGE REFS
POST /api/snapshots

Create snapshot from list of package references.

This API creates snapshot out of any list of package references. Package references could be obtained from other snapshots, local repos or mirrors.

update README image with greener look

--debug, --username, --server, --password appears to be broken

Looks like global options in aptly_command.rb are not working presently.

debug_output $stdout L66 in aptly_command.rb doesn't appear to respond and even if you set debug_output $stdout directly in class AptlyCommand it isn't used when running http queries in let's say aptly_repo

$ aptly-cli repo_list --debug --no-config
{"Name"=>"testrepo", "Comment"=>"", "DefaultDistribution"=>"", "DefaultComponent"=>"main"}
{"Name"=>"testrepocreate", "Comment"=>"testing repo creation", "DefaultDistribution"=>"precisecreatetest", "DefaultComponent"=>""}
{"Name"=>"testrepotoshow", "Comment"=>"testing repo show", "DefaultDistribution"=>"preciseshowtest", "DefaultComponent"=>""}
{"Name"=>"testrepo20", "Comment"=>"", "DefaultDistribution"=>"", "DefaultComponent"=>"main"}
{"Name"=>"testrepotoquery", "Comment"=>"testing repo query with name", "DefaultDistribution"=>"precisequerytest", "DefaultComponent"=>""}
{"Name"=>"testrepoedit", "Comment"=>"testing repo edit distro name", "DefaultDistribution"=>"preciseeditdistnew", "DefaultComponent"=>""}

The above command should include HTTP debug output from HTTParty.

Incorrect encoding on Ruby 1.9.1

Seeing this on Ruby 1.9.1.

# aptly-cli 
/usr/local/bin/aptly-cli:23:in `load': /var/lib/gems/1.9.1/gems/aptly_cli-0.1.5/bin/aptly-cli:28: invalid multibyte char (US-ASCII) (SyntaxError)
/var/lib/gems/1.9.1/gems/aptly_cli-0.1.5/bin/aptly-cli:28: invalid multibyte char (US-ASCII)
/var/lib/gems/1.9.1/gems/aptly_cli-0.1.5/bin/aptly-cli:28: syntax error, unexpected $end, expecting keyword_end
... would be created if it doesn’t exist.'
...                               ^
    from /usr/local/bin/aptly-cli:23:in `<main>'

Solution is to add the following to the aptly-cli script just below #!/usr/bin/env ruby

#!/usr/bin/env ruby
# encoding: utf-8

Nonintuitive config for /api not at server root

My aptly api is behind a reverse proxy that exposes it at http://example.com:8888/aptly/api. After a lot of fiddling, I was able to get aptly_cli to eventually connect to it. Unfortunately, the behavior is non intuitive because of the way the base uri is constructed:
https://github.com/sepulworld/aptly_cli/blob/master/lib/aptly_command.rb#L55

Instead of having a config file with:
:servername: example.com/aptly
:port: 8888

it was necessary to specify the config as:
:servername: example.com
:port: 8888/aptly

It seems like an extra optional configuration argument that specifies the root prefix (/aptly) or constructing the base url in a different way that recognizes example.com/aptly and inserts the port after .com would be helpful.