markwarneke / az.cli

Python azure.cli.core wrapper to execute Azure CLI commands using Python3

Hi there ✌️

📫 markwarneke.me • 🐤 @MarkWarneke

🥚 Co-Founder @mmmint.ai - TypeScript Nest.js & Vue.js | Python FastAPI

az.cli's Issues

[Bug] az login blocks

Repro

Running the login command blocks the execution. The logs prompting to authenticate is not returned until the method is canceled.

az('login')

After canceling:

AzResult(exit_code=1, out={}, log='To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code XXXX  to authenticate.\n')

Expected Behaviour:

• The log should be returned so that the user can do the interactive login.
• An error should be raised that interactive login is not permitted.

Code Reference

ttps://github.com/MarkWarneke/Az.Cli/blob/ab287f40278855cd3b6c9bc28c573c0ed910f99e/src/az/cli.py#L71

Based on an E-Mail regarding multiple Azure Config Files

Describe the bug

I work with multiple tenants / accounts and to make my life easier I use the following configuration:

BASH_FUNC_az_uar%%=() { AZURE_CONFIG_DIR=/home/user/.azure_uat az "$@"
BASH_FUNC_az_prd%%=() { AZURE_CONFIG_DIR=/home/user/.azure_prd az "$@"
BASH_FUNC_az_sit%%=() { AZURE_CONFIG_DIR=/home/user/.azure_sit az "$@"

So I can run the commands without having to log in to each tenant!

az_uat ad app show --id XXXXXXXXXX
az_prd ad app show --id XXXXXXXXXX
az_sit ad app show --id XXXXXXXXXX

Unfortunately in a script when trying to run the commands in different tenants I got authentication errors. I tried to adjust the AZURE_CONFIG_DIR variable before loading the module, I also tried using importlib and reloading the module but nothing worked!

As a last solution, I started using the subprocess module and ran each command in a Python interpreter and got the value using a Queue. That way it worked!

But as a curiosity, I would like to know if there is a way to adjust AZURE_CONFIG_DIR programmatically from code!

To Reproduce

Create multiple az context

az login

mv ~/.azure/* ~/.azure-mw
az login --service-principal -u $id -p $p -t $t

# Validate this works
AZURE_CONFIG_DIR=.azure az group list 
AZURE_CONFIG_DIR=.azure-mw az group list 

Expected behavior

When changing the environment variable AZURE_CONFIG_DIR the command should not fail and use the at the point in time configured Azure config.

az fails in multiple threads

Greetings!

az works great in one thread, but fails in 2 and more. I am just executing an innocent command, it always fails in 2, 3 threads.

    cmd = 'ad signed-in-user show --query "userPrincipalName"'
    logger.info(f'will run: {cmd}')
    exit_code, result, logs = az(cmd)

Exception in thread aks-stage-3se:
Traceback (most recent call last):
  File "/usr/local/Cellar/[email protected]/3.8.6/Frameworks/Python.framework/Versions/3.8/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/local/Cellar/[email protected]/3.8.6/Frameworks/Python.framework/Versions/3.8/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "mainA.py", line 211, in process_cluster
    cluster_summary()
  File "mainA.py", line 178, in cluster_summary
    exit_code, result, logs = az(cmd)
  File "/Users/vorobyov001/tre/PycharmProjects/011-key-vault/az-ad/zvenv/lib/python3.8/site-packages/az/cli.py", line 89, in az
    exit_code = _cli.invoke(
  File "/Users/vorobyov001/tre/PycharmProjects/011-key-vault/az-ad/zvenv/lib/python3.8/site-packages/knack/cli.py", line 233, in invoke
    self.raise_event(EVENT_CLI_POST_EXECUTE)
  File "/Users/vorobyov001/tre/PycharmProjects/011-key-vault/az-ad/zvenv/lib/python3.8/site-packages/knack/cli.py", line 164, in raise_event
    func(self, **kwargs)
  File "/Users/vorobyov001/tre/PycharmProjects/011-key-vault/az-ad/zvenv/lib/python3.8/site-packages/azure/cli/core/azlogging.py", line 180, in deinit_cmd_metadata_logging
    cli_ctx.logging.end_cmd_metadata_logging(cli_ctx.result.exit_code if cli_ctx.result else 128)
  File "/Users/vorobyov001/tre/PycharmProjects/011-key-vault/az-ad/zvenv/lib/python3.8/site-packages/azure/cli/core/azlogging.py", line 190, in end_cmd_metadata_logging
    self.command_metadata_logger.removeHandler(self.command_logger_handler)
AttributeError: 'NoneType' object has no attribute 'removeHandler'

--only-show-errors not honored on commands

When using --only-show-errors on commands to suppress the 'in preview' (and other) warnings the warnings are still shown in the logger and therefore do not get suppressed

[Feature] pin project dependencies.

Is your feature request related to a problem? Please describe.
As this package depends heavily on azure-cli-core the dependency should be pinned on a particular major version which is working.

For now a nightly build checks if the basic test suit is still running. This however could easily break if a major version is released.

Also publishing of the module is done using a non fixed dependency.

Describe the solution you'd like

See https://stackoverflow.com/questions/28509481/should-i-pin-my-python-dependencies-versions

Describe alternatives you've considered

Alternatively the test suit could be extended.
I am wondering if the modules of the az is effected, what happens if a particular module breaks - how is the dependency handled within azure-cli-core?

JSONDecoderError: AZURE_SUBSCRIPTION_ID = az('account list --query "[?isDefault][id]" --all -o tsv')

Describe the bug
AZURE_SUBSCRIPTION_ID = az('account list --query "[?isDefault][id]" --all -o tsv') does not seem to work inside Python

To Reproduce
Steps to reproduce the behavior:

Expected behavior
In setenv.sh:

#!/bin/sh
AZURE_SUBSCRIPTION_ID=$(az account list --query "[?isDefault][id]" --all -o tsv)
echo ${AZURE_SUBSCRIPTION_ID}

4d9XXXX-XXXX-XXXXXX-XXXXXXXXXXXX

But inside test.py:

AZURE_SUBSCRIPTION_ID = az('account list --query "[?isDefault][id]" --all -o tsv')
Exception has occurred: JSONDecodeError
Extra data: line 1 column 2 (char 1)

Desktop (please complete the following information):

• OS: Mac OSX
• Browser chrome,

Additional context
So how do we capture the stdout to screen as az commands output to screen only and it is not returned as results to be tested? i.e

account list --query "[?isDefault][id]" --all -o tsv

az account list -o table

ad sp create-for-rbac --name servicePrincipalArthurPAF > aks4seb1app
{
"appId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"displayName": "servicePrincipalID,
"password": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"tenant": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
}

Disable logging of library

Is your feature request related to a problem? Please describe.
When I import az.cli and logging, logging show the log of az.cli as well.
Describe the solution you'd like
An option or property to disable logs of az.cli. I think there should be an if command at these lines

Az.Cli/src/az/cli.py

Lines 67 to 75 in c69eed8

	root_logger = logging.getLogger()
	root_handler = logging.StreamHandler(stream=log_buf)
	root_handler.setLevel(logging.CRITICAL)
	root_logger.addHandler(root_handler)
	# cli logger keeps WARNING and above
	cli_logger = logging.getLogger(CLI_LOGGER_NAME)
	cli_handler = logging.StreamHandler(stream=log_buf)
	cli_handler.setLevel(logging_level)

Additional context

adding az.cli to requirments.txt causes ancient azure packages to be installed.

Describe the bug
Adding az.cli to requirments.txt causes ancient azure packages to be installed.

To Reproduce
This is my requrirments.txt file, when az.cli is uncommented the problem occurs and other azure packages are stuck at very old versions unless they are manually set to new ones by specifying a version:

crate
urllib3
requests
azure-identity
azure-keyvault-secrets
azure-storage-blob
azure.cosmos
azure-cli
ipykernel
tqdm
#az.cli