aiven / aiven-client

aiven-client (avn) is the official command-line client for Aiven

Aiven Client

Aiven is a next-generation managed cloud services platform. Its focus is in ease of adoption, high fault resilience, customer's peace of mind and advanced features at competitive price points. See https://aiven.io/ for more information about the backend service.

aiven-client (avn) is the official command-line client for Aiven.

Contents

• Getting Started
  • Install from PyPi
  • Build an RPM Package
  • Check Installation
  • Log In
• Usage
  • Authenticate: Logins and Tokens
  • Choose your Cloud
  • Working with Projects
  • Explore Existing Services
  • Launch Services
  • Managing service users
  • Service Integrations
  • Custom Files
  • Working with Teams
  • Configuring OAuth2 Clients
• Extra Features
  • Autocomplete
  • Auth Helpers
• Contributing
• Keep Reading

Getting Started

Requirements:

• Python 3.8 or later
• Requests
• For Windows and OSX, certifi is also needed

Install from PyPi

Pypi installation is the recommended route for most users:

$ python3 -m pip install aiven-client

Build an RPM Package

It is also possible to build an RPM:

$ make rpm

Check Installation

To check that the tool is installed and working, run it without arguments:

$ avn

If you see usage output, you're all set.

Note: On Windows you may need to use python3 -m aiven.client instead of avn.

Log In

The simplest way to use Aiven CLI is to authenticate with the username and password you use on Aiven:

$ avn user login <[email protected]>

The command will prompt you for your password.

You can also use an access token generated in the Aiven Console:

$ avn user login <[email protected]> --token

You will be prompted for your access token as above.

If you are registered on Aiven through the AWS or GCP marketplace, then you need to specify an additional argument --tenant. Currently the supported value are aws and gcp, for example:

$ avn user login <[email protected]> --tenant aws

Usage

Some handy hints that work with all commands:

• The avn help command shows all commands and can search for a command, so for example avn help kafka topic shows commands with kafka and topic in their description.
• Passing -h or --help gives help output for any command. Examples: avn --help or avn service --help.
• All commands will output the raw REST API JSON response with --json, we use this extensively ourselves in conjunction with jq.

Authenticate: Logins and Tokens

Login:

$ avn user login <[email protected]>

Logout (revokes current access token, other sessions remain valid):

$ avn user logout

Expire all authentication tokens for your user, logs out all web console sessions, etc. You will need to login again after this:

$ avn user tokens-expire

Manage individual access tokens:

$ avn user access-token list
$ avn user access-token create --description <usage_description> [--max-age-seconds <secs>] [--extend-when-used]
$ avn user access-token update <token|token_prefix> --description <new_description>
$ avn user access-token revoke <token|token_prefix>

Note that the system has hard limits for the number of tokens you can create. If you're permanently done using a token you should always use user access-token revoke operation to revoke the token so that it does not count towards the quota.

Alternatively, you can add 2 JSON files, first create a default config in ~/.config/aiven/aiven-credentials.json containing the JSON with an auth_token:

{
    "auth_token": "ABC1+123...TOKEN==",
    "user_email": "[email protected]"
}

Second create a default config in ~/.config/aiven/aiven-client.json containing the json with the default_project:

{"default_project": "yourproject-abcd"}

Choose your Cloud

List available cloud regions:

$ avn cloud list

Working with Projects

List projects you are a member of:

$ avn project list

Project commands operate on the currently active project or the project specified with the --project NAME switch. The active project cab be changed with the project switch command:

$ avn project switch <projectname>

Show active project's details:

$ avn project details

Create a project and set the default cloud region for it:

$ avn project create myproject --cloud aws-us-east-1

Delete an empty project:

$ avn project delete myproject

List authorized users in a project:

$ avn project user-list

Invite an existing Aiven user to a project:

$ avn project user-invite [email protected]

Remove a user from the project:

$ avn project user-remove [email protected]

View project management event log:

$ avn events

Explore Existing Services

List services (of the active project):

$ avn service list

List services in a specific project:

$ avn service list --project proj2

List only a specific service:

$ avn service list db1

Verbose list (includes connection information, etc.):

$ avn service list db1 -v

Full service information in JSON, as it is returned by the Aiven REST API:

$ avn service list db1 --json

Only a specific field in the output, custom formatting:

$ avn service list db1 --format "The service is at {service_uri}"

View service log entries (most recent entries and keep on following logs, other options can be used to get history):

$ avn service logs db1 -f

Launch Services

View available service plans:

$ avn service plans

Launch a PostgreSQL service:

$ avn service create mydb -t pg --plan hobbyist

View service type specific options, including examples on how to set them:

$ avn service types -v

Launch a PostgreSQL service of a specific version (see above command):

$ avn service create mydb96 -t pg --plan hobbyist -c pg_version=9.6

Update a service's list of allowed client IP addresses. Note that a list of multiple values is provided as a comma separated list:

$ avn service update mydb96 -c ip_filter=10.0.1.0/24,10.0.2.0/24,1.2.3.4/32

Open psql client and connect to the PostgreSQL service (also available for InfluxDB):

$ avn service cli mydb96

Update a service to a different plan AND move it to another cloud region:

$ avn service update mydb --plan startup-4 --cloud aws-us-east-1

Power off a service:

$ avn service update mydb --power-off

Power on a service:

$ avn service update mydb --power-on

Terminate a service (all data will be gone!):

$ avn service terminate mydb

Managing service users

Some service types support multiple users (e.g. PostgreSQL database users).

List, add and delete service users:

$ avn service user-list
$ avn service user-create
$ avn service user-delete

For Valkey services it's possible to create users with ACLs:

$ avn service user-create --username new_user --valkey-acl-keys="prefix* another_key" --valkey-acl-commands="+set" --valkey-acl-categories="-@all +@admin" --valkey-acl-channels="prefix* some_chan" my-valkey-service

Service users are created with strong random passwords.

Service Integrations

Service integrations allow to link Aiven services to other Aiven services or to services offered by other companies for example for logging. Some examples for various diffenent integrations: Google cloud logging, AWS Cloudwatch logging, Remote syslog integration and Getting started with Datadog.

List service integration endpoints:

$ avn service integration-endpoint-list

List all available integration endpoint types for given project:

$ avn service integration-endpoint-types-list --project <project>

Create a service integration endpoint:

$ avn service integration-endpoint-create --project <project> --endpoint-type <endpoint type> --endpoint-name <endpoint name> --user-config-json <user configuration as json>
$ avn service integration-endpoint-create --project <project> --endpoint-type <endpoint type> --endpoint-name <endpoint name> -c <KEY=VALUE type user configuration>

Update a service integration endpoint:

$ avn service integration-endpoint-update --project <project> --user-config-json <user configuration as json> <endpoint id>
$ avn service integration-endpoint-update --project <project> -c <KEY=VALUE type user configuration> <endpoint id>

Delete a service integration endpoint:

$ avn service integration-endpoint-delete --project <project>  <endpoint_id>

List service integrations:

$ avn service integration-list <service name>

List all available integration types for given project:

$ avn service integration-types-list --project <project>

Create a service integration:

$ avn service integration-create --project <project> -t <integration type> -s <source service> -d <dest service> -S <source endpoint id> -D <destination endpoint id> --user-config-json <user configuration as json>
$ avn service integration-create --project <project> -t <integration type> -s <source service> -d <dest service> -S <source endpoint id> -D <destination endpoint id> -c <KEY=VALUE type user configuration>

Update a service integration:

$ avn service integration-update --project <project> --user-config-json <user configuration as json> <integration_id>
$ avn service integration-update --project <project> -c <KEY=VALUE type user configuration> <integration_id>

Delete a service integration:

$ avn service integration-delete --project <project> <integration_id>

Custom Files

Listing files:

$ avn service custom-file list --project <project> <service_name>

Reading file:

$ avn service custom-file get --project <project> --file_id <file_id> [--target_filepath <file_path>] [--stdout_write] <service_name>

Uploading new files:

$ avn service custom-file upload --project <project> --file_type <file_type> --file_path <file_path> --file_name <file_name> <service_name>

Updating existing files:

$ avn service custom-file update --project <project> --file_path <file_path> --file_id <file_id> <service_name>

Working with Teams

List account teams:

$ avn account team list <account_id>

Create a team:

$ avn account team create --team-name <team_name> <account_id>

Delete a team:

$ avn account team delete --team-id <team_id> <account_id>

Attach team to a project:

$ avn account team project-attach --team-id <team_id> --project <project_name> <account_id> --team-type <admin|developer|operator|read_only>

Detach team from project:

$ avn account team project-detach --team-id <team_id> --project <project_name> <account_id>

List projects associated to the team:

$ avn account team project-list --team-id <team_id> <account_id>

List members of the team:

$ avn account team user-list --team-id <team_id> <account_id>

Invite a new member to the team:

$ avn account team user-invite --team-id <team_id> <account_id> <[email protected]>

See the list of pending invitations:

$ avn account team user-list-pending --team-id <team_id> <account_id>

Remove user from the team:

$ avn account team user-delete --team-id <team_id> --user-id <user_id> <account_id>

Configuring OAuth2 Clients

List configured OAuth2 clients:

$ avn account oauth2-client list <account_id>

Get a configured OAuth2 client's configuration:

$ avn account oauth2-client list <account_id> --oauth2-client-id <client_id>

Create a new OAuth2 client information:

$ avn account oauth2-client create <account_id> --name <app_name> -d <app_description> --redirect-uri <redirect_uri>

Delete an OAuth2 client:

$ avn account oauth2-client delete <account_id> --oauth2-client-id <client_id>

List an OAuth2 client's redirect URIs:

$ avn account oauth2-client redirect-list <account_id> --oauth2-client-id <client_id>

Create a new OAuth2 client redirect URI:

$ avn account oauth2-client redirect-create <account_id> --oauth2-client-id <client_id> --redirect-uri <redirect_uri>

Delete an OAuth2 client redirect URI:

$ avn account oauth2-client redirect-delete <account_id> --oauth2-client-id <client_id> --redirect-uri-id <redirect_uri_id>

List an OAuth2 client's secrets:

$ avn account oauth2-client secret-list <account_id> --oauth2-client-id <client_id>

Create a new OAUth2 client secret:

$ avn account oauth2-client secret-create <account_id> --oauth2-client-id <client_id>

Delete an OAuth2 client's secret:

$ avn account oauth2-client secret-delete <account_id> --oauth2-client-id <client_id> --secret-id <secret_id>

Extra Features

Autocomplete

avn supports shell completions. It requires an optional dependency: argcomplete. Install it:

$ python3 -m pip install argcomplete

To use completions in bash, add following line to ~/.bashrc:

eval "$(register-python-argcomplete avn)"

For more information (including completions usage in other shells) see https://kislyuk.github.io/argcomplete/.

Auth Helpers

When you spin up a new service, you'll want to connect to it. The --json option combined with the jq utility is a good way to grab the fields you need for your specific service. Try this to get the connection string:

$ avn service get --json <service> | jq ".service_uri"

Each project has its own CA cert, and other services (notably Kafka) use mutualTLS so you will also need the service.key and service.cert files too for those. Download all three files to the local directory:

$ avn service user-creds-download --username avnadmin <service>

For working with kcat (see also our help article ) or the command-line tools that ship with Kafka itself, a keystore and trustore are needed. By specifying which user's creds to use, and a secret, you can generate these via avn too:

$ avn service user-kafka-java-creds --username avnadmin -p t0pS3cr3t <service>

Contributing

Check the CONTRIBUTING guide for details on how to contribute to this repository.

Keep Reading

We maintain some other resources that you may also find useful:

• Command Line Magic with avn
• Managing Billing Groups via CLI

aiven-client's Issues

Running avn

What can we help you with?

I installed aiven-client on macos m1 Ventura 13.1

Running avn :

avn
ERROR:root:code for hash md5 was not found.
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type md5
ERROR:root:code for hash sha1 was not found.
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha1
ERROR:root:code for hash sha224 was not found.
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha224
ERROR:root:code for hash sha256 was not found.
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha256
ERROR:root:code for hash sha384 was not found.
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha384
ERROR:root:code for hash sha512 was not found.
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha512
Traceback (most recent call last):
  File "/usr/local/bin/avn", line 7, in <module>
    from aiven.client.__main__ import main
  File "/usr/local/lib/python2.7/site-packages/aiven/client/__init__.py", line 6, in <module>
    from .client import AivenClient  # noqa
  File "/usr/local/lib/python2.7/site-packages/aiven/client/client.py", line 19, in <module>
    import requests
  File "/usr/local/lib/python2.7/site-packages/requests/__init__.py", line 43, in <module>
    import urllib3
  File "/usr/local/lib/python2.7/site-packages/urllib3/__init__.py", line 8, in <module>
    from .connectionpool import (
  File "/usr/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 29, in <module>
    from .connection import (
  File "/usr/local/lib/python2.7/site-packages/urllib3/connection.py", line 38, in <module>
    from .util.ssl_ import (
  File "/usr/local/lib/python2.7/site-packages/urllib3/util/__init__.py", line 6, in <module>
    from .ssl_ import (
  File "/usr/local/lib/python2.7/site-packages/urllib3/util/ssl_.py", line 8, in <module>
    from hashlib import md5, sha1, sha256
ImportError: cannot import name md5
 

python -V
Python 3.8.5

Python2 is also installed

Where would you expect to find this information?

API Deprecated

Hi,

I'm trying to retrieve logs:

avn logs --project <project>
ERROR	command failed: Error: {"errors":[{"message":"API deprecated, switch to /project/<project>/service/<service_name>/logs endpoint","status":400}],"message":"API deprecated, switch to /project/<project>/service/<service_name>/logs endpoint"}

Thank you

Tests fail while linting with C901 (cyclomatic complexity)

What happened?

Linting started failing on python 3.10 with:

+ make test
make[3]: Entering directory '/var/home/vladan/rpmbuild/BUILD/aiven-client'
fatal: not a git repository (or any parent up to mount point /var/home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /var/home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /var/home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
python3 -m flake8 aiven tests
aiven/client/client.py:1619:5: C901 'AivenClient.create_project' is too complex (12)
aiven/client/client.py:1675:5: C901 'AivenClient.update_project' is too complex (12)
aiven/client/client.py:1854:5: C901 'AivenClient.create_billing_group' is too complex (13)
aiven/client/client.py:1908:5: C901 'AivenClient.update_billing_group' is too complex (14)
aiven/client/cli.py:205:5: C901 'AivenCLI.create_user_config' is too complex (13)
aiven/client/cli.py:846:5: C901 'AivenCLI.service__plans' is too complex (12)
aiven/client/cli.py:1840:5: C901 'AivenCLI.service__user_creds_download' is too complex (11)
aiven/client/cli.py:3743:5: C901 'AivenCLI.service__create' is too complex (11)
aiven/client/cli.py:3979:5: C901 'AivenCLI.service__update' is too complex (12)
aiven/client/argx.py:212:5: C901 'CommandLineTool.print_response' is too complex (13)
aiven/client/pretty.py:86:1: C901 'yield_table' is too complex (25)
aiven/client/connection_info/_utils.py:25:1: C901 'format_uri' is too complex (11)
make[3]: *** [Makefile:30: flake8] Error 1
make[3]: Leaving directory '/var/home/vladan/rpmbuild/BUILD/aiven-client'
error: Bad exit status from /var/tmp/rpm-tmp.YPXhnf (%check)

Cyclomatic complexity is something that needs to be sorted out in the listed functions.

What did you expect to happen?

Tests should have passed.

What else do we need to know?

Include your platform, version, and any other information that seems relevant.

Fedora 35
Python 3.10

Change default to `--no-project-vpc` when creating a service

What is currently missing?

When I create a service using the console, it defaults to "Public Internet".

When I create a service using avn, it defaults to using a VPC is there is one available. For instance I observed this with:

avn project switch <project>
avn service create tibs-demo-pg --service-type pg --cloud google-europe-north1 --plan hobbyist

When I then try to connect to the service CLI:

avn service cli tibs-demo-pg

I get:

psql: error: could not translate host name "tibs-demo-pg-dev-sandbox.aivencloud.com" to address: nodename nor servname provided, or not known

(and the same result if I use psql directly).

This was entirely surprising to me, as I'd expect the console and CLI to behave the same way when creating a service.

I can work around this by knowing what the problem is, and then migrating the service.
For reference of anyone reading this, that means:

1. Go to the Cloud and VPC section on the service page on the console
2. Check that it says "Project VPC"
3. Select Migrate Cloud
4. Choose a service from the appropriate tab (for instance, Europe) instead of the VPC tab
5. Migrate

How could this be improved?

Change the default behaviour to pass project_vpc_id: null, as if the user had specified --no-project-vpc.

This is an incompatible change, so would need to be called out in the release notes.

Is this a feature you would work on yourself?

[x] I plan to open a pull request for this feature

Access token passed with --token flag gets revoked after logout command

What happened?

Configuring automation to login with a token, using following command

/ # avn user login [email protected] --token
[email protected]'s Aiven access token:
INFO	Aiven credentials written to: /root/.config/aiven/aiven-credentials.json
INFO	Default project set as 'test' (change with 'avn project switch <project>')

After other avn commands are successfully executed, logging out

/ # avn user logout

Token gets revoked.

aiven-client/aiven/client/cli.py

Lines 419 to 422 in e224b2a

	def user__logout(self):
	"""Logout from current session"""
	self.client.access_token_revoke(token_prefix=self._get_auth_token())
	self._remove_auth_token_file()

What did you expect to happen?

Token passed during login with --token flag to not be revoked.

What else do we need to know?

Using machine user and its token for automation, currently only terraform provider. Trying to automate creation Java keystore and truststore with avn service user-kafka-java-creds. I did not expect automation token to be revoked after logout.

/ # avn --version
aiven-client 2.14.5

v.2.5.0: `integration-endpoint-types-list` and `integration-types-list` throw errors

$ avn --version
aiven-client 2.5.0

$ avn service integration-endpoint-types-list
Traceback (most recent call last):
  File "/usr/local/bin/avn", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/aiven/client/__main__.py", line 5, in main
    AivenCLI().main()
  File "/usr/local/lib/python3.7/site-packages/aiven/client/argx.py", line 296, in main
    sys.exit(self.run())
  File "/usr/local/lib/python3.7/site-packages/aiven/client/argx.py", line 268, in run
    return self.run_actual(args)
  File "/usr/local/lib/python3.7/site-packages/aiven/client/argx.py", line 290, in run_actual
    return func()  # pylint: disable=not-callable
  File "/usr/local/lib/python3.7/site-packages/aiven/client/cli.py", line 1324, in service__integration_endpoint_types_list
    endpoint_types = self.client.get_service_integration_endpoint_types(self.args.project)
  File "/usr/local/lib/python3.7/site-packages/aiven/client/client.py", line 303, in get_service_integration_endpoint_types
    path = self.build_path("project", project, "integration_endpoint_types")
  File "/usr/local/lib/python3.7/site-packages/aiven/client/client.py", line 167, in build_path
    return "/" + "/".join(quote(part, safe="") for part in parts)
  File "/usr/local/lib/python3.7/site-packages/aiven/client/client.py", line 167, in <genexpr>
    return "/" + "/".join(quote(part, safe="") for part in parts)
  File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/parse.py", line 834, in quote
    return quote_from_bytes(string, safe)
  File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/parse.py", line 859, in quote_from_bytes
    raise TypeError("quote_from_bytes() expected bytes")
TypeError: quote_from_bytes() expected bytes

$ avn service integration-types-list
Traceback (most recent call last):
  File "/usr/local/bin/avn", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/aiven/client/__main__.py", line 5, in main
    AivenCLI().main()
  File "/usr/local/lib/python3.7/site-packages/aiven/client/argx.py", line 296, in main
    sys.exit(self.run())
  File "/usr/local/lib/python3.7/site-packages/aiven/client/argx.py", line 268, in run
    return self.run_actual(args)
  File "/usr/local/lib/python3.7/site-packages/aiven/client/argx.py", line 290, in run_actual
    return func()  # pylint: disable=not-callable
  File "/usr/local/lib/python3.7/site-packages/aiven/client/cli.py", line 1410, in service__integration_types_list
    endpoint_types = self.client.get_service_integration_types(self.args.project)
  File "/usr/local/lib/python3.7/site-packages/aiven/client/client.py", line 337, in get_service_integration_types
    path = self.build_path("project", project, "integration_types")
  File "/usr/local/lib/python3.7/site-packages/aiven/client/client.py", line 167, in build_path
    return "/" + "/".join(quote(part, safe="") for part in parts)
  File "/usr/local/lib/python3.7/site-packages/aiven/client/client.py", line 167, in <genexpr>
    return "/" + "/".join(quote(part, safe="") for part in parts)
  File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/parse.py", line 834, in quote
    return quote_from_bytes(string, safe)
  File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/parse.py", line 859, in quote_from_bytes
    raise TypeError("quote_from_bytes() expected bytes")
TypeError: quote_from_bytes() expected bytes

'pg' is the only service type with a shortnened name

More of a ''nice to have"" feature.

When you run

$  avn service types

pg is the only service with a shorthand variant.

For the sake of consistency/and or command-line brevity, could each of these other services have a shorthand variant? Instinctively, I typed in es when creating an Elasticsearch service.

No kafka topic configuration while using topic-get

What can we help you with?

While querying kafka topic information using topic-get, the "topic.config" values are not shown.
https://api.aiven.io/doc/#tag/Service:_Kafka/operation/ServiceKafkaTopicGet

Would it be possible to also include some of the topic configurations? e.g:
segment_bytes
segment_ms
retention_ms

PyPi pip install aiven-client appears to be broken

Does not work: python3 -m pip install aiven-client

05:40:53 (venv_ok) jmunsch@pop-os heroku-buildpack-aiven-deploy ±|master ✗|→ python3 -m pip install aiven-client
Collecting aiven-client
  Using cached https://files.pythonhosted.org/packages/30/f0/67d40e7a2091a95e72dec23d6627a994c8111869e8f732a2d8fc2ca0e2e2/aiven-client-2.3.2.tar.gz
    ERROR: Command errored out with exit status 1:
     command: /home/jmunsch/PycharmProjects/propertymeld/venv_ok/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-i2r96qi3/aiven-client/setup.py'"'"'; __file__='"'"'/tmp/pip-install-i2r96qi3/aiven-client/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info
         cwd: /tmp/pip-install-i2r96qi3/aiven-client/
    Complete output (7 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-i2r96qi3/aiven-client/setup.py", line 45, in <module>
        version=version.get_project_version("aiven/client/version.py"),
      File "/tmp/pip-install-i2r96qi3/aiven-client/version.py", line 34, in get_project_version
        % version_file)
    Exception: version not available from git or from file '/tmp/pip-install-i2r96qi3/aiven-client/aiven/client/version.py'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
WARNING: You are using pip version 19.2.3, however version 19.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

These all fail with the same error:

2301  08/01/20 17:45:48 python3 -m pip install https://github.com/aiven/aiven-client/archive/2.2.3.tar.gz
 2302  08/01/20 17:47:32 python3 -m pip install https://github.com/aiven/aiven-client/archive/2.3.2.zip
 2303  08/01/20 17:47:50 python3 -m pip install https://github.com/aiven/aiven-client/archive/2.3.1.zip
 2304  08/01/20 17:47:58 python3 -m pip install https://github.com/aiven/aiven-client/archive/2.3.0.zip
 2305  08/01/20 17:48:07 python3 -m pip install https://github.com/aiven/aiven-client/archive/2.2.5.zip
 2306  08/01/20 17:48:19 python3 -m pip install https://github.com/aiven/aiven-client/archive/2.2.4.zip

Works: python3 -m pip install git+https://github.com/aiven/aiven-client.git

05:50:37 (venv_ok) jmunsch@pop-os propertymeld ±|APP-520-jm-1 ✗|→ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Pop!_OS 18.04 LTS
Release:	18.04
Codename:	bionic

05:52:56 (venv_ok) jmunsch@pop-os propertymeld ±|APP-520-jm-1 ✗|→ python3 --version
Python 3.6.7

AVN: Create integration of a Aiven solution but the destination service is in another project

I want to create a service integration with a Aiven solution: Monitor Logs in OpenSearch
To give some context. I'm trying to activate the Monitor Logs in OpenSearch of PostgreSQL instances. This OpenSearch instance is in a different project than the PostgreSQL instance.
In the console, when I try to create the integration, it shoes 2 options to choose from: "Project name" and "Service name". Then I choose the other project and select the service name.

But when I try to use the avn client (v 4.0.0), I can't.
Executing the command: avn service integration-create -h
I have this options:

options:
  -h, --help            show this help message and exit
  --project PROJECT     Project name to use, default None
  -t INTEGRATION_TYPE, --integration-type INTEGRATION_TYPE
                        Integration type
  -s SOURCE_SERVICE, --source-service SOURCE_SERVICE
                        Source service name
  -d DEST_SERVICE, --dest-service DEST_SERVICE
                        Destination service name
  -S SOURCE_ENDPOINT_ID, --source-endpoint-id SOURCE_ENDPOINT_ID
                        Source integration endpoint id
  -D DEST_ENDPOINT_ID, --dest-endpoint-id DEST_ENDPOINT_ID
                        Destination integration endpoint id
  -c KEY=VALUE          Apply a configuration setting. See 'avn service types -v' for available values.
  --user-config-json USER_CONFIG_JSON
                        JSON string or path (preceded by '@') to a JSON configuration file
  --json                Raw json output

So, the command should be something like this:
avn service integration-create --project source_project_test -s source_service_test --dest-service destination_service_test -t logs --user-config-json '{"elasticsearch_index_days_max": 3, "elasticsearch_index_prefix": "logs"}'

But I have the error saying: Destination service 'destination_service_test' does not exist"

I think it gives this error because it is searching in the same project. How can I set a different project for the destionation? I've tried "--dest-project" but this option doesn't exist (error: unrecognized arguments: --dest-project).

Am I missing something? Or I can't create this native integration using AVN client?
I know I have the option to choose external source OpenSearch and point to the existing Aiven instance in a different project. But I would prefer to have the native solution.

Add contributing file

What can we help you with?

Hey 🙋🏻‍♀️, do you think is a good idea to add a contributing file with the steps in how to get started contributing to this repository?
I think it makes sense to have some quick HowTo:

• run the tests
• debugging steps
• how to run
• about commit messages
• issues
• others

Where would you expect to find this information?

It should have it's own file and be linked to the readme of the repository

Logs API deprecated

After successfully logged, the bellow error is showed when I try to read logs from our project:

$ avn logs -f prod
ERROR	command failed: Error: {"errors":[{"message":"API deprecated, switch to /project/<project>/service/<service_name>/logs endpoint","status":400}],"message":"API deprecated, switch to /project/<project>/service/<service_name>/logs endpoint"}

Version: aiven-client (2.1.3-8-g76a0b5d)

AttributeError: 'Namespace' object has no attribute 'name'

Version:

aiven_client-2.5.0_10_g923391b-py3-none-any.whl

Attempted command:

avn --auth-token "###" service create --project xxx-review-apps --service-type pg --plan startup-4 --cloud aws-us-east-1 -c pg_version=11 xxx-pr-2978 

Error:

Traceback (most recent call last):
  File "/app/.heroku/python/bin/avn", line 8, in <module>
    sys.exit(main())
  File "/app/.heroku/python/lib/python3.6/site-packages/aiven/client/__main__.py", line 5, in main
    AivenCLI().main()
  File "/app/.heroku/python/lib/python3.6/site-packages/aiven/client/argx.py", line 298, in main
    sys.exit(self.run())
  File "/app/.heroku/python/lib/python3.6/site-packages/aiven/client/argx.py", line 270, in run
    return self.run_actual(args)
  File "/app/.heroku/python/lib/python3.6/site-packages/aiven/client/argx.py", line 292, in run_actual
    return func()  # pylint: disable=not-callable
  File "/app/.heroku/python/lib/python3.6/site-packages/aiven/client/cli.py", line 2730, in service__create
    service=self.args.name,

AttributeError: 'Namespace' object has no attribute 'name'

Any suggestions?

In the meanwhile we've rolled back to 2.3.5

Error quote_from_bytes()

What happened?

Ran the following command and got the error
avn --auth-token $AIVEN_USER_TOKEN service list

Traceback (most recent call last):
  File "/usr/bin/avn", line 8, in <module>
    sys.exit(main())
  File "/usr/lib/python3.8/site-packages/aiven/client/__main__.py", line 5, in main
    AivenCLI().main()
  File "/usr/lib/python3.8/site-packages/aiven/client/argx.py", line 309, in main
    sys.exit(self.run(args))
  File "/usr/lib/python3.8/site-packages/aiven/client/argx.py", line 281, in run
    return self.run_actual(args)
  File "/usr/lib/python3.8/site-packages/aiven/client/argx.py", line 303, in run_actual
    return func()  # pylint: disable=not-callable
  File "/usr/lib/python3.8/site-packages/aiven/client/cli.py", line 966, in service__list
    services = self.client.get_services(project=self.get_project())
  File "/usr/lib/python3.8/site-packages/aiven/client/client.py", line 1546, in get_services
    self.build_path("project", project, "service"),
  File "/usr/lib/python3.8/site-packages/aiven/client/client.py", line 168, in build_path
    return "/" + "/".join(quote(part, safe="") for part in parts)
  File "/usr/lib/python3.8/site-packages/aiven/client/client.py", line 168, in <genexpr>
    return "/" + "/".join(quote(part, safe="") for part in parts)
  File "/usr/lib/python3.8/urllib/parse.py", line 851, in quote
    return quote_from_bytes(string, safe)
  File "/usr/lib/python3.8/urllib/parse.py", line 876, in quote_from_bytes
    raise TypeError("quote_from_bytes() expected bytes")
TypeError: quote_from_bytes() expected bytes

What did you expect to happen?

A list of all services

What else do we need to know?

This command fails on the following system: Alpine Linux 3.12.0

> python3 --version
Python 3.8.10
> avn --version
aiven-client 2.14.4

This command run successfully on Ubuntu 20.04, with same version of python3 and aiven-client

Missing CONTRIBUTING.md/.rst and/or DEVELOPMENT.md/rst guidelines

Hi all. I stumbled upon this today and I find it quite useful (thanks for that!).

I started to have a look at your APIs and I was willing to send some PRs to add more functionality to the client but I found no CONTRIBUTING.md (or .rst) and/or DEVELOPMENT.md (or .rst) guidelines as part of this project. Is this expected?

What should outside collaborators wanting to contribute do? What guidelines should they follow?

I personally always use pipenv for local development, so I was thinking of adding an initial DEVELOPMENT.rst with the local development setup instructions in there.

Would that work for you?

Thanks in advance.

Add type annotations

What is currently missing?

Type annotations (and type checks) are missing.

How could this be improved?

Adding type annotations is beneficial for improving reliability and code maintainability. Annotations can be added gradually, module by module. Of course, these annotations should be checked using the type checker. The most popular type checker is MyPy; let's use it.

• Introduce mypy (#259)
• aiven/__init__.py (was already annotated)
• aiven/client/__init__.py (was already annotated)
• aiven/client/__main__.py (#259)
• aiven/client/argx.py (#272)
• aiven/client/cli.py (#286)
• aiven/client/cliarg.py (#278)
• aiven/client/client.py (#276)
• aiven/client/common.py (#270)
• aiven/client/connection_info/ (#278)
• aiven/client/envdefault.py (#270)
• aiven/client/pretty.py (#281)
• aiven/client/session.py (#270)
• aiven/client/speller.py (#279)
• aiven/client/version.py (#270)
• tests/__init__.py (#283)
• tests/test_argx.py (#283)
• tests/test_cli.py (#283)
• tests/test_cliarg.py (#283)
• tests/test_pretty.py (#281)
• tests/test_session.py (#283)
• tests/test_speller.py (#279)
• version.py (#289)
• Enable type checks without ignores

Is this a feature you would work on yourself?

• I plan to open a pull request for this feature

More intuitive 'natural' language for avn client

If it were possible to, as per #92, set up some simple aliases or wrapper functions to do something like this, e.g.

#! usr/bin/bash

alias='avn count events'="avn events > events.txt && wc -l events.txt"
alias='avn list services'="avn service list"
alias='avn list clouds'="avn cloud list"
...

I'm starting a shell script to catch some useful aliases/alternate command names where possible.

Add --version argument

It would be useful to have a --version argument for avn. Currently, if I want to know which client version I'm using, I have to execute pip show aiven-client.

"Current service plan does not allow pinning a pool to a single user. Please contact support if you need this functionality

What can we help you with?

What does this mean? And how does one go about getting pools in front of review app instances?

Input:

avn --auth-token "###" service connection-pool-create asdf-pr-3488 --project asdf-review-apps --dbname defaultdb --username asdf --pool-name asdf-pool --pool-size 50 --json 

Output:

ERROR command failed: Error: {"errors":[{"message":"Current service plan does not allow pinning a pool to a single user. Please contact support if you need this functionality.","status":403}],"message":"Current service plan does not allow pinning a pool to a single user. Please contact support if you need this functionality."}

startup-4
aws-us-east-1

https://github.com/Property-Meld/heroku-buildpack-aiven-deploy-pg/blob/master/bin/tasks.py#L203

Where would you expect to find this information?

• google
• aiven docs
• support
• github

Kafka topic function arguments not aligned with API Spec

What happened?

Using the AivenClient functions for kafka topic handling (creation and updating) I expected a similar interface as in the HTTP API documentation (https://api.aiven.io/doc/#operation/ServiceKafkaTopicCreate, https://api.aiven.io/doc/#operation/ServiceKafkaTopicUpdate)

But the function arguments of AivenClient.create_service_topic and AivenClient.update_service_topic are very different:

Not all config parameters are exposed, and the exposed ones are deprecated since they should be in a config dict (namely min_insync_replicas, retention_bytes, retention_hours, cleanup_policy)

Also most of the parameters are mandatory, while the API lists any but topic_name as optional

What did you expect to happen?

Having the function signature in sync with the backing API, since they are basically only passed as a body to a generic client

What else do we need to know?

Versions:
aiven-client 2.15.0
API version v1

Missing "response example" in download PDF invoice REST API documentation

I am currently implementing the ability to download an invoice PDF (expect PR soon!) but I noticed the download PDF invoice REST API endpoint is not documented in detail (link).

The Response Example present in other REST API endpoints is missing. Is this expected?

Thank you

Service database username avnadmin does not exist status 403

What happened?

• pools are not being created
• Service database username avnadmin does not exist","status":403

avn --auth-token "###" service connection-pool-create asdf-pr-3586 --project asdf-review-apps --dbname defaultdb --username avnadmin --pool-name asdf-pool --pool-size 50 --json 

ERROR	command failed: Error: {"errors":

[{"message":"Service database username avnadmin does not exist","status":403}],"message":"Service database username avnadmin does not exist"}

Traceback (most recent call last):
  File "/app/.heroku/python/bin/invoke", line 8, in <module>
    sys.exit(program.run())
  File "/app/.heroku/python/lib/python3.8/site-packages/invoke/program.py", line 384, in run
    self.execute()
  File "/app/.heroku/python/lib/python3.8/site-packages/invoke/program.py", line 566, in execute
    executor.execute(*self.tasks)
  File "/app/.heroku/python/lib/python3.8/site-packages/invoke/executor.py", line 129, in execute
    result = call.task(*args, **call.kwargs)
  File "/app/.heroku/python/lib/python3.8/site-packages/invoke/tasks.py", line 127, in __call__
    result = self.body(*args, **kwargs)
  File "/tmp/codon/tmp/buildpacks/47e2e265d630dfd66344f130d506a6039afb0919/bin/tasks.py", line 368, in create_pool_uri_and_set_env
    pool_uri = create_pool(config)
  File "/tmp/codon/tmp/buildpacks/47e2e265d630dfd66344f130d506a6039afb0919/bin/tasks.py", line 203, in create_pool
    do_popen(pool_create_cmd(**config), err_msg="Failed to create pool.")
  File "/tmp/codon/tmp/buildpacks/47e2e265d630dfd66344f130d506a6039afb0919/bin/tasks.py", line 132, in do_popen
    raise exc(err_msg)
Exception: Failed to create pool.

What did you expect to happen?

Pool to get created

What else do we need to know?

• https://github.com/Property-Meld/heroku-buildpack-aiven-deploy-pg
• https://github.com/Property-Meld/heroku-buildpack-aiven-deploy-pg/blob/master/bin/tasks.py#L30
• https://github.com/Property-Meld/heroku-buildpack-aiven-deploy-pg/blob/master/bin/tasks.py#L203

No Kafka Connect or Schema Registry functionality

I'm sure this is on your roadmap, but I just wanted to note that it'd be very nice to have the Kafka Connect and Schema Registry API endpoints accessible via the CLI.

Output error on empty resultset

What happened?

If the pretty module is called with headers, but without any results having a value for a given header, it errors out as no width is calculated (see

aiven-client/aiven/client/pretty.py

Line 147 in 3934dd6

if header:

).

What did you expect to happen?

Column printed with just the width of the header value.

What else do we need to know?

Include your platform, version, and any other information that seems relevant.

'avn service create' fails on case-sensitive service name

$ avn service create myEs -t elasticsearch --plan hobbyist

<Response [400]>

ERROR    command failed: Error: {"errors":[{"message":"Service name must consist of 1-64 alphanumerical characters","status":400}],"message":"Service name must consist of 1-64 alphanumerical characters"}

TypeError getting topic info

When getting topic info, for some topic the consumer groups aren't listed but there is an error instead.

avn service topic-get --project tuup kafka-tuup dev-order-events-ingest-1
PARTITION  ISR  SIZE       EARLIEST_OFFSET  LATEST_OFFSET  GROUPS
=========  ===  =========  ===============  =============  ======
0          2    xxx    xxx                xxx           30
1          2    xxx   xxx           xxx         30
2          2    xxx  xxx          xxx        30

Traceback (most recent call last):
  File "/usr/local/python/versions/3.7.2/bin/avn", line 11, in <module>
    load_entry_point('aiven-client==2.1.7', 'console_scripts', 'avn')()
  File "/usr/local/python/versions/3.7.2/lib/python3.7/site-packages/aiven/client/__main__.py", line 5, in main
    AivenCLI().main()
  File "/usr/local/python/versions/3.7.2/lib/python3.7/site-packages/aiven/client/argx.py", line 246, in main
    sys.exit(self.run())
  File "/usr/local/python/versions/3.7.2/lib/python3.7/site-packages/aiven/client/argx.py", line 223, in run
    return self.run_actual(args)
  File "/usr/local/python/versions/3.7.2/lib/python3.7/site-packages/aiven/client/argx.py", line 240, in run_actual
    return func()  # pylint: disable=not-callable
  File "/usr/local/python/versions/3.7.2/lib/python3.7/site-packages/aiven/client/cli.py", line 1009, in service_topic_get
    "lag": p["latest_offset"] - cg["offset"]
TypeError: unsupported operand type(s) for -: 'int' and 'NoneType'

[Docs] Minor suggestion for ensuring good quality commit messages

What is currently missing?

I noticed that in your CONTRIBUTING guide you reference Chris Beam's blogpost.
However there is currently no way to encourage/enforce these suggested practices.

How could this be improved?

A simple solution would be to use a commit message template.
It can be part of this repository. Here's how it could work:

• We would add a new file called .gitmessage
• It would contain the template below.
• The CONTRIBUTING.md would contain the following

With git command :

$ git config commit.template .gitmessage

Or by adding in the config file the following lines :

[commit]
  template = .gitmessage

Git commit message template .gitmessage

# Title: Summary, imperative, start upper case, don't end with a period
# No more than 50 chars. #### 50 chars is here: #

# Remember blank line between title and body.

# Body: Explain *what* and *why* (not *how*). Include task ID (Jira issue).
# Wrap at 72 chars. ################################## which is here: #


# At the end: Include Co-authored-by for all contributors. 
# Include at least one empty line before it. Format: 
# Co-authored-by: name <[email protected]>
#
# How to Write a Git Commit Message:
# https://chris.beams.io/posts/git-commit/
#
# 1.Separate subject from body with a blank line
# 2. Limit the subject line to 50 characters
# 3. Capitalize the subject line
# 4. Do not end the subject line with a period
# 5. Use the imperative mood in the subject line
# 6. Wrap the body at 72 characters
# 7. Use the body to explain what and why vs. how
EOF

Is this a feature you would work on yourself?

[X] I plan to open a pull request for this feature

Running tests fail on Python 3.10

Running tests fail on Python 3.10

aiven-client on  link-contributing-to-readme [?] via 🐍 v3.10.0 (python-3.10) 
❯ make test
python3 -m flake8 aiven tests
python3 -m pylint aiven tests
************* Module aiven.client.session
aiven/client/session.py:12:39: E1136: Value 'Optional' is unsubscriptable (unsubscriptable-object)
aiven/client/session.py:22:37: E1136: Value 'Optional' is unsubscriptable (unsubscriptable-object)
************* Module aiven.client.cli
aiven/client/cli.py:135:76: E1136: Value 'Optional' is unsubscriptable (unsubscriptable-object)
aiven/client/cli.py:3846:84: E1136: Value 'Optional' is unsubscriptable (unsubscriptable-object)
************* Module aiven.client.argx
aiven/client/argx.py:126:37: E1136: Value 'Optional' is unsubscriptable (unsubscriptable-object)
************* Module aiven.client.connection_info._utils
aiven/client/connection_info/_utils.py:28:14: E1136: Value 'Optional' is unsubscriptable (unsubscriptable-object)
aiven/client/connection_info/_utils.py:29:14: E1136: Value 'Optional' is unsubscriptable (unsubscriptable-object)
aiven/client/connection_info/_utils.py:30:10: E1136: Value 'Optional' is unsubscriptable (unsubscriptable-object)
aiven/client/connection_info/_utils.py:31:10: E1136: Value 'Optional' is unsubscriptable (unsubscriptable-object)
aiven/client/connection_info/_utils.py:32:12: E1136: Value 'Optional' is unsubscriptable (unsubscriptable-object)
aiven/client/connection_info/_utils.py:34:11: E1136: Value 'Optional' is unsubscriptable (unsubscriptable-object)

-----------------------------------
Your code has been rated at 9.85/10

make: *** [pylint] Error 2
┌───────────────────────────────────────────────────────────────────────────────
(python-3.10) 

This is not a question more like just for your information. So not sure if should be an issue or not. Just leaving it here.

`avn service logs` doesn't allow pagination

I ran avn service logs --project project-name service-name expecting to get a lot of logs. It did not go far back enough so I had to work out how to paginate/offset the returned values.

I can see I can set -n (limit) to up to 500 according to the documentation, but that doesn't go far back enough. The documentation indicates an offset can be passed, which I expect is the mechanism for searching back through log history, but the aiven-client does not expose this configuration.

Can you add an -o (offset) flag, or some other way of allowing iteration through log history? Thanks.

Also, as a bonus, as a user I really don't care what an offset is, or what the internal organisation of the logs is, I just want to see logs in chronological order (ascending or descending).

Help text for USER_CONFIG is misleading

In the help text for 'avn service update' it has:

-c USER_CONFIG User configuration: KEY=JSON_VALUE

This is fairly misleading since for arrays it actually takes a comma separated string (https://github.com/aiven/aiven-client/blob/master/aiven/client/cli.py#L33).

Also, probably wouldn't hurt to have an example of updating the ip whitelist in the docs (for postgres)?

avn service update my_service -c ip_filter='10.0.0.1/24,10.0.0.2/24'

SSL error `SSLV3_ALERT_BAD_RECORD_MAC` on a fresh Mac install

What happened?

After installing the avn CLI by running pip install aiven-client on a Mac (as per https://developer.aiven.io/docs/tools/cli.html), I get the following error when trying to run commands like avn project list or avn account list):

SSLError(SSLError(1, '[SSL: SSLV3_ALERT_BAD_RECORD_MAC] sslv3 alert bad record mac (_ssl.c:1125)'

What did you expect to happen?

I expected to see a list of my projects or accounts.

What else do we need to know?

This seems to be a problem related to the out-of-the-box SSL implementation on Macs (LibreSSL). I solved it locally by installing OpenSSL through homebrew and then using a homebrew version of python using the following steps:

1. brew install openssl
2. Make sure the $PATH contains the relevant parts from homebrew at the front (mine is now PATH=/opt/homebrew/opt/openssl@3/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin)
3. brew reinstall python (which puts python3 and pip3 into /opt/homebrew/bin but doesn't replace the local python)
4. pip3 install aiven-client

After those steps, I have /opt/homebrew/bin/avn available which works fine.

Add examples for tagging projects and services

What can we help you with?

I'd love to see some examples of #228 in action and which objects I can tag.

Where would you expect to find this information?

Either in the README of this project, or with the other docs https://developer.aiven.io/docs/tools/cli

user-kafka-java-creds - avax.crypto.BadPaddingException

What happened?

Credentials downloaded thanks to avn service user-kafka-java-creds --project xxxx xxxx --username avnadmin seems corrupted.

I used the files and default properties (changeit...) downloaded.

2022-01-07 15:59:00,810 ERROR [io.sma.rea.mes.provider] (main) SRMSG00230: Unable to create the publisher or subscriber during initialization: org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
	at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:823)
	at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:665)
	at io.smallrye.reactive.messaging.kafka.impl.ReactiveKafkaConsumer.<init>(ReactiveKafkaConsumer.java:80)
	at io.smallrye.reactive.messaging.kafka.impl.KafkaSource.<init>(KafkaSource.java:90)
	at io.smallrye.reactive.messaging.kafka.KafkaConnector.getPublisherBuilder(KafkaConnector.java:193)
	at io.smallrye.reactive.messaging.kafka.KafkaConnector_ClientProxy.getPublisherBuilder(KafkaConnector_ClientProxy.zig:159)
	at io.smallrye.reactive.messaging.impl.ConfiguredChannelFactory.createPublisherBuilder(ConfiguredChannelFactory.java:190)
	at io.smallrye.reactive.messaging.impl.ConfiguredChannelFactory.register(ConfiguredChannelFactory.java:153)
	at io.smallrye.reactive.messaging.impl.ConfiguredChannelFactory.initialize(ConfiguredChannelFactory.java:125)
	at io.smallrye.reactive.messaging.impl.ConfiguredChannelFactory_ClientProxy.initialize(ConfiguredChannelFactory_ClientProxy.zig:189)
	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
	at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
	at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
	at io.smallrye.reactive.messaging.extension.MediatorManager.start(MediatorManager.java:189)
	at io.smallrye.reactive.messaging.extension.MediatorManager_ClientProxy.start(MediatorManager_ClientProxy.zig:220)
	at io.quarkus.smallrye.reactivemessaging.runtime.SmallRyeReactiveMessagingLifecycle.onApplicationStart(SmallRyeReactiveMessagingLifecycle.java:41)
	at io.quarkus.smallrye.reactivemessaging.runtime.SmallRyeReactiveMessagingLifecycle_Observer_onApplicationStart_4e8937813d9e8faff65c3c07f88fa96615b70e70.notify(SmallRyeReactiveMessagingLifecycle_Observer_onApplicationStart_4e8937813d9e8faff65c3c07f88fa96615b70e70.zig:111)
	at io.quarkus.arc.impl.EventImpl$Notifier.notifyObservers(EventImpl.java:322)
	at io.quarkus.arc.impl.EventImpl$Notifier.notify(EventImpl.java:304)
	at io.quarkus.arc.impl.EventImpl.fire(EventImpl.java:73)
	at io.quarkus.arc.runtime.ArcRecorder.fireLifecycleEvent(ArcRecorder.java:128)
	at io.quarkus.arc.runtime.ArcRecorder.handleLifecycleEvents(ArcRecorder.java:97)
	at io.quarkus.deployment.steps.LifecycleEventsBuildStep$startupEvent1144526294.deploy_0(LifecycleEventsBuildStep$startupEvent1144526294.zig:87)
	at io.quarkus.deployment.steps.LifecycleEventsBuildStep$startupEvent1144526294.deploy(LifecycleEventsBuildStep$startupEvent1144526294.zig:40)
	at io.quarkus.runner.ApplicationImpl.doStart(ApplicationImpl.zig:711)
	at io.quarkus.runtime.Application.start(Application.java:101)
	at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:101)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:66)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:42)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:119)
	at io.quarkus.runner.GeneratedMain.main(GeneratedMain.zig:29)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at io.quarkus.bootstrap.runner.QuarkusEntryPoint.doRun(QuarkusEntryPoint.java:48)
	at io.quarkus.bootstrap.runner.QuarkusEntryPoint.main(QuarkusEntryPoint.java:25)
Caused by: org.apache.kafka.common.KafkaException: Failed to load SSL keystore /etc/config/kafka.keystore.p12 of type PKCS12
	at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedStore.load(DefaultSslEngineFactory.java:377)
	at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedStore.<init>(DefaultSslEngineFactory.java:349)
	at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.createKeystore(DefaultSslEngineFactory.java:299)
	at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.configure(DefaultSslEngineFactory.java:161)
	at org.apache.kafka.common.security.ssl.SslFactory.instantiateSslEngineFactory(SslFactory.java:140)
	at org.apache.kafka.common.security.ssl.SslFactory.configure(SslFactory.java:97)
	at org.apache.kafka.common.network.SslChannelBuilder.configure(SslChannelBuilder.java:73)
	at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:192)
	at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:81)
	at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:105)
	at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:737)
	... 36 more
Caused by: java.io.IOException: keystore password was incorrect
	at java.base/sun.security.pkcs12.PKCS12KeyStore.engineLoad(PKCS12KeyStore.java:2159)
	at java.base/sun.security.util.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:221)
	at java.base/java.security.KeyStore.load(KeyStore.java:1473)
	at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedStore.load(DefaultSslEngineFactory.java:374)
	... 46 more
Caused by: java.security.UnrecoverableKeyException: failed to decrypt safe contents entry: javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.

relevant error seems to be this one : java.security.UnrecoverableKeyException: failed to decrypt safe contents entry: javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption

When i use openssl and keytool with PEM files my consumer start without issue.

What did you expect to happen?

I expect my consumer to works with credentials provided.

What else do we need to know?

macOS 12.0.1

avn --version
aiven-client 2.12.0

TypeError powering down service

When attempting to power down a service, an error gets thrown.

$ avn service update dev-bus --power-off

Result:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.7.5/x64/bin/avn", line 11, in <module>
    load_entry_point('aiven-client==2.3.5', 'console_scripts', 'avn')()
  File "/opt/hostedtoolcache/Python/3.7.5/x64/lib/python3.7/site-packages/aiven/client/__main__.py", line 5, in main
    AivenCLI().main()
  File "/opt/hostedtoolcache/Python/3.7.5/x64/lib/python3.7/site-packages/aiven/client/argx.py", line 259, in main
    sys.exit(self.run())
  File "/opt/hostedtoolcache/Python/3.7.5/x64/lib/python3.7/site-packages/aiven/client/argx.py", line 231, in run
    return self.run_actual(args)
  File "/opt/hostedtoolcache/Python/3.7.5/x64/lib/python3.7/site-packages/aiven/client/argx.py", line 253, in run_actual
    return func()  # pylint: disable=not-callable
  File "/opt/hostedtoolcache/Python/3.7.5/x64/lib/python3.7/site-packages/aiven/client/cli.py", line 2272, in service__update
    service = self.client.get_service(project=project, service=self.args.name)
  File "/opt/hostedtoolcache/Python/3.7.5/x64/lib/python3.7/site-packages/aiven/client/client.py", line 176, in get_service
    return self.verify(self.get, self.build_path("project", project, "service", service), result_key="service")
  File "/opt/hostedtoolcache/Python/3.7.5/x64/lib/python3.7/site-packages/aiven/client/client.py", line 156, in build_path
    return "/" + "/".join(quote(part, safe="") for part in parts)
  File "/opt/hostedtoolcache/Python/3.7.5/x64/lib/python3.7/site-packages/aiven/client/client.py", line 156, in <genexpr>
    return "/" + "/".join(quote(part, safe="") for part in parts)
  File "/opt/hostedtoolcache/Python/3.7.5/x64/lib/python3.7/urllib/parse.py", line 834, in quote
    return quote_from_bytes(string, safe)
  File "/opt/hostedtoolcache/Python/3.7.5/x64/lib/python3.7/urllib/parse.py", line 859, in quote_from_bytes
    raise TypeError("quote_from_bytes() expected bytes")
TypeError: quote_from_bytes() expected bytes`

issue printing logs with non-ascii characters

  File "/usr/local/bin/avn", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/site-packages/aiven/client/__main__.py", line 5, in main
    AivenCLI().main()
  File "/usr/local/lib/python2.7/site-packages/aiven/client/argx.py", line 246, in main
    sys.exit(self.run())
  File "/usr/local/lib/python2.7/site-packages/aiven/client/argx.py", line 223, in run
    return self.run_actual(args)
  File "/usr/local/lib/python2.7/site-packages/aiven/client/argx.py", line 240, in run_actual
    return func()  # pylint: disable=not-callable
  File "/usr/local/lib/python2.7/site-packages/aiven/client/cli.py", line 245, in service_logs
    new_offset = self._show_logs(msgs)
  File "/usr/local/lib/python2.7/site-packages/aiven/client/cli.py", line 213, in _show_logs
    print("{time:<27}{hostname} {unit} {msg}".format(**log_msg))
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0130' in position 452: ordinal not in range(128)

Character it blows up on: https://www.fileformat.info/info/unicode/char/0130/index.htm

cli: add support for deleting pending invites : when a new release

What can we help you with?

this PR is merged #312

when a new release of aiven cli (nothing since 1rst of July)

Where would you expect to find this information?

Add pagination to `avn service log`

What is currently missing?

Currently we can get up to 500 logs entries but there is no way to see previous logs. In our case, we need to get logs of few hours / few days before, to diagnose some issues we have.

How could this be improved?

• add a param to service log to query for a given offset (but the offset will always change, when logs entries arrive, so this is not the best but already good)
• allow to query logs by start / end dates (but probably hard for you as there seems to be no API for it)

Is this a feature you would work on yourself?

No

[ ] I plan to open a pull request for this feature

Error when downloading user creds for Postgres

aiven client version: 2.2.3

When running command

avn service user-creds-download --project my-project my-pg-service --username myusername

I get the error

Traceback (most recent call last):
  File "/usr/local/bin/avn", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/aiven/client/__main__.py", line 5, in main
    AivenCLI().main()
  File "/usr/local/lib/python3.7/site-packages/aiven/client/argx.py", line 262, in main
    sys.exit(self.run())
  File "/usr/local/lib/python3.7/site-packages/aiven/client/argx.py", line 234, in run
    return self.run_actual(args)
  File "/usr/local/lib/python3.7/site-packages/aiven/client/argx.py", line 256, in run_actual
    return func()  # pylint: disable=not-callable
  File "/usr/local/lib/python3.7/site-packages/aiven/client/cli.py", line 789, in service__user_creds_download
    fp.write(user["access_cert"])
KeyError: 'access_cert'

avn logout fails on Mac OSX

Tried

$  avn user logout 

and got

ERROR	command failed: Error: 404: Not Found

Add billing_email_address and tech_email_address

Hello,

Could you please add in this CLI, the option to add/update the billing_email_address and tech_email_address of a Aiven project

As avn project update ..

Thank you in advance,

Benjamin

simple counter for 'avn events' (and other discrete outputs)

```avn events`` returns basically an event log.

I can imagine an admin possibly wanting an easy, built-in function that counts the numbers of events by user.

Here's the clumsy way I currently do this:

avn events > events && wc -l events
     102 events

argx should preserve signature of wrapped function

In

aiven-client/aiven/client/argx.py

Line 88 in 46f2cfd

return func

, we don't cast back to the signature of the wrapped function, but we should, as well as appending the type of the added args.

See https://mypy.readthedocs.io/en/stable/generics.html#declaring-decorators.