mludvig / aws-ssm-tools Goto Github PK

View Code? Open in Web Editor NEW

225.0 225.0 34.0 188 KB

Handy tools for AWS Systems Manager - ssm-session, ecs-session, ssm-ssh and ssm-tunnel

License: Other

Python 90.84% Shell 3.27% HCL 5.89%

aws aws-ssm ecs ip ssh tunneling

aws-ssm-tools's People

Contributors

Stargazers

Watchers

aws-ssm-tools's Issues

Alignment not allowed error when running ssm-session --list

I've noticed that if there is no tag called Name on an instance this error is thrown:

Traceback (most recent call last):
  File "/usr/local/bin/ssm-session", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/ssm_tools/ssm_session_cli.py", line 81, in main
    InstanceResolver(args).print_list()
  File "/usr/local/lib/python3.7/site-packages/ssm_tools/resolver.py", line 103, in print_list
    print(f"{item['InstanceId']}   {item['HostName']:{hostname_len}}   {item['InstanceName']:{instname_len}}   {' '.join(item['Addresses'])}")
ValueError: '=' alignment not allowed in string format specifier

Might be helpful to have a more informative error message.

tunnel with fewer deps

Hi Michael! Thanks for some very cool tools
Unfortunately on OSX don't have the "ip" command.
Also since I'd be ok not having my local routing changed, and port-forward, connecting to local port like you get with e.g. aws ssm --document-name AWS-StartPortForwardingSession ....

I tested this out with socat + the stock AWS Document, and can do something like:
socat -v TCP4-LISTEN:9999,fork,reuseaddr TCP4:172.16.8.56:80

and am thinking, for my use case, I might take your code for ssm-tunnel, do the local forward as described, and replace the agent with socat.

I guess I could also look at making my own SSM Document just like the AWS one, but extend it to (install+) run socat on the remote ...

It's quite a change and perhaps not necessarily an improvement on the existing method, just interested if you had any thoughts or advice there.
Cheers
Nick

Getting timeout on establishing a tunnel

Hi,
Do you know why running something like this in Ubuntu 20.04 (python 3.8) always fails with timeout?
Thanks

ivan@anecua:~$ ssm-tunnel i-7171717171717171 --profile my-profile --debug
[ssm-tunnel] DEBUG: Logging level set to DEBUG
[ssm-tunnel] DEBUG: Spawning: aws --profile my-profile  ssm start-session --target i-7171717171717171
[ssm-tunnel] DEBUG: PID: 288300
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 111, in expect_loop
    incoming = spawn.read_nonblocking(spawn.maxread, timeout)
  File "/usr/lib/python3/dist-packages/pexpect/pty_spawn.py", line 482, in read_nonblocking
    raise TIMEOUT('Timeout exceeded.')
pexpect.exceptions.TIMEOUT: Timeout exceeded.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/ssm-tunnel", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/dist-packages/ssm_tools/ssm_tunnel_cli.py", line 345, in main
    tunnel = SsmTunnel(instance_id, profile=args.profile, region=args.region, logger_name=logger_name)
  File "/usr/local/lib/python3.8/dist-packages/ssm_tools/ssm_tunnel_cli.py", line 83, in __init__
    super().__init__(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/ssm_tools/talker.py", line 9, in __init__
    self.connect(instance_id, profile, region)
  File "/usr/local/lib/python3.8/dist-packages/ssm_tools/talker.py", line 23, in connect
    self.wait_for_prompt()
  File "/usr/local/lib/python3.8/dist-packages/ssm_tools/talker.py", line 49, in wait_for_prompt
    self._child.expect('sh.*\$ $')
  File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 340, in expect
    return self.expect_list(compiled_pattern_list,
  File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 369, in expect_list
    return exp.expect_loop(timeout)
  File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 119, in expect_loop
    return self.timeout(e)
  File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 82, in timeout
    raise TIMEOUT(msg)
pexpect.exceptions.TIMEOUT: Timeout exceeded.
<pexpect.pty_spawn.spawn object at 0x7f81cf756100>
command: /usr/local/bin/aws
args: [b'/usr/local/bin/aws', b'--profile', b'my-profile', b'ssm', b'start-session', b'--target', b'i-7171717171717171']
buffer (last 100 chars): '\r\nStarting session with SessionId: my-user@company-097fbec8ab0500248\r\n$ '
before (last 100 chars): '\r\nStarting session with SessionId: my-user@company-097fbec8ab0500248\r\n$ '
after: <class 'pexpect.exceptions.TIMEOUT'>
match: None
match_index: None
exitstatus: None
flag_eof: False
pid: 288300
child_fd: 5
closed: False
timeout: 10
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
searcher: searcher_re:
    0: re.compile('sh.*\\$ $')

`ec2-ssh` fails against non-EC2 managed instances

I have a few Raspberry Pis that are registered as managed instances mi-* in Fleet Manager. Connecting to them works fine with a command like ec2-session pi-1:

$ ec2-session pi-1
Starting session with SessionId: strophy-0f0424c62fea5e575
$ 
Exiting session with sessionId: strophy-0f0424c62fea5e575.

But ec2-ssh fails with the following behaviour:

$ ec2-ssh pi-1
INFO: Resolved instance name 'pi-1' to 'mi-048afb7f6460a110d'
ERROR: Unable to figure out the EC2 login name. Use "-l {user}" or {user}@{instance}.
$ ec2-ssh pi@pi-1
INFO: Resolved instance name 'pi-1' to 'mi-048afb7f6460a110d'
INFO: Using SSH key from SSH Agent, should be as good as any.
ERROR: An error occurred (InvalidArgsException) when calling the SendSSHPublicKey operation: 1 validation error detected: Value 'mi-048afb7f6460a110d' at 'instanceId' failed to satisfy constraint: Member must satisfy regular expression pattern: ^i-[a-f0-9]+$

Is this a restriction imposed by aws-ssm-tools or the underlying ssm start-session command? Are there any workarounds?

No Licence File

I would like to use this in my organization but I am not able to as there is no licence file included which details the terms of use. Something along the lines of: https://www.apache.org/licenses/LICENSE-2.0#apply or whatever licence that you want to use.

ecs-session list fails if one ECS cluster has no tasks

I encountered some problems to use ecs-session in one of my account.
After adding --debug arg, it appeared that one of my ECS cluster has no tasks (seems not so useful, but why not).

After digging a little in the code, the issue happens here :
It fails when using the paginator list_tasks that returns an empty list of tasks.

Error thrown is

[ssm-tools.ecs-session] ERROR: An error occurred (InvalidParameterException) when calling the DescribeTasks operation: Tasks cannot be empty.

It would be better to first check if the paginator is empty or not, before going further.
However, I'm not sure what is the best way to check :

use boto3 can_paginate
or via KeyCount key on the first page (as mentionned here

Docs: Password for ssm-user

Hi, thanks for this great package!

The docs could explain authentication a little bit more. Namely, it wasn't clear to me that the password I am asked for by ssm-tunnel is my passowd on my local machine (though I am sure it is obvious to many). It would be nice to mention that explicitly. Thank you!

[feat] Add shortcut to run command via ssm-session

Hello :-)
Thanks for providing these tools, very useful!
I have a feature suggestion:

Current way to run a custom command:

ssm-session my-instance --document-name AWS-StartInteractiveCommand --parameters 'command=["echo 42"]'

I would suggest a new option --command to simplify this use case:

ssm-session my-instance --command "echo 42"

The new option would be incompatible with --document-name and --parameters.
The --user option could be made compatible with --command (just concatenate sudo -i -u USER with COMMAND).

What do you think about this?
If you find it useful, I could try submitting a PR.
This feature would be even more useful once this AWS CLI / session-manager-plugin issue is fixed: aws/amazon-ssm-agent#358

ssm-ssh does not work on Windows 10

When attempting to connect to an SSM managed instance with ssm-ssh command on Windows 10 using latest OpenSSH build, this error is received from ssm-ssh

SSM inventory entity not recognised: {'Id': '<Instance ID>', 'Data': {}}
CreateProcessW failed error:2
posix_spawnp: No such file or directory

Possible fix is per microsoft/vscode-remote-release#18 (comment), this should be resolved by running ssh.exe on Windows instead of ssh.

ssh with proxycommand doesn't work

when I do:
ssh i-xyz

I get this error:
Pseudo-terminal will not be allocated because stdin is not a terminal.
-bash: line 1: $'SSH-2.0-OpenSSH_8.1\r': command not found

I use this ssh config:
host i-* mi-*
ProxyCommand ssm-ssh --profile prd --region eu-west-1 remoteuser@%h

Use access key environment variables instead of profile?

Is it possible to set an AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in the environment instead of using a profile? I assume it should work since you're just invoking the AWS CLI under the hood, but all the documentation seems to expect that you declare a profile name to provide credentials.

Cannot copy files, get 'Timeout exceeded'

This utility is great, but ssm-copy is not working for me:

➜  ~ ssm-copy test i-05bxxxxxxxxxxx:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/pexpect/expect.py", line 111, in expect_loop
    incoming = spawn.read_nonblocking(spawn.maxread, timeout)
  File "/usr/local/lib/python3.7/site-packages/pexpect/pty_spawn.py", line 482, in read_nonblocking
    raise TIMEOUT('Timeout exceeded.')
pexpect.exceptions.TIMEOUT: Timeout exceeded.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/ssm-copy", line 204, in <module>
    main()
  File "/usr/local/bin/ssm-copy", line 186, in main
    sender = SsmFileSender(instance_id, profile = args.profile, region = args.region, logger_name = logger_name)
  File "/usr/local/lib/python3.7/site-packages/ssm_tools/talker.py", line 9, in __init__
    self.connect(instance_id, profile, region)
  File "/usr/local/lib/python3.7/site-packages/ssm_tools/talker.py", line 23, in connect
    self.wait_for_prompt()
  File "/usr/local/lib/python3.7/site-packages/ssm_tools/talker.py", line 49, in wait_for_prompt
    self._child.expect('sh.*\$ $')
  File "/usr/local/lib/python3.7/site-packages/pexpect/spawnbase.py", line 341, in expect
    timeout, searchwindowsize, async_)
  File "/usr/local/lib/python3.7/site-packages/pexpect/spawnbase.py", line 369, in expect_list
    return exp.expect_loop(timeout)
  File "/usr/local/lib/python3.7/site-packages/pexpect/expect.py", line 119, in expect_loop
    return self.timeout(e)
  File "/usr/local/lib/python3.7/site-packages/pexpect/expect.py", line 82, in timeout
    raise TIMEOUT(msg)
pexpect.exceptions.TIMEOUT: Timeout exceeded.
<pexpect.pty_spawn.spawn object at 0x10952a390>
command: /usr/local/bin/aws
args: [b'/usr/local/bin/aws', b'ssm', b'start-session', b'--target', b'i-05b7c0ffb1bc9664a']
buffer (last 100 chars): '\r\nStarting session with SessionId: 1569835485940510000-0723a5e801a6f63f0\r\n$ '
before (last 100 chars): '\r\nStarting session with SessionId: 1569835485940510000-0723a5e801a6f63f0\r\n$ '
after: <class 'pexpect.exceptions.TIMEOUT'>
match: None
match_index: None
exitstatus: None
flag_eof: False
pid: 35112
child_fd: 7
closed: False
timeout: 10
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
searcher: searcher_re:
    0: re.compile('sh.*\\$ $')

ssm-session to the same instance works fine.

The instance is Debian but I'm on Mac OS locally. Installed aws-ssm-tools using pip3. My python installation is:

#!/usr/local/opt/python/bin/python3.7

aws-cli and session-manager-plugin versions:

aws --version
aws-cli/1.16.248 Python/3.7.3 Darwin/18.7.0 botocore/1.12.238

session-manager-plugin --version
1.1.33.0

Thanks

Warning lines for instances not in inventory

This warning can be quite annoying if you have many instances not in inventory. It shows not just on a list but also ssh-ssm utils etc

$ ssm-session --list --profile admin-guideitprod
[ssm-session] WARNING: SSM inventory entity not recognised: {'Id': 'i-02a#############c4b', 'Data': {}}
....

I changed the warning to debug temporarily maybe you want to make that permanent. Or if better option let me know.

$ diff /usr/local/lib/python3.8/dist-packages/ssm_tools/resolver.py /tmp/resolver.py
39a40
>
61,62c62
<                     #logger.warning("SSM inventory entity not recognised: %s", entity)
<                     logger.debug("SSM inventory entity not recognised: %s", entity)
---
>                     logger.warning("SSM inventory entity not recognised: %s", entity)

ssm-tools is not working for managed instances(Instance ID starting with mi-)

ssm-session --list --profile SBX-POC
Traceback (most recent call last):
File "/usr/local/bin/ssm-session", line 11, in
sys.exit(main())
File "/usr/local/lib/python3.6/dist-packages/ssm_tools/ssm_session_cli.py", line 81, in main
InstanceResolver(args).print_list()
File "/usr/local/lib/python3.6/dist-packages/ssm_tools/resolver.py", line 88, in print_list
items = self.get_list().values()
File "/usr/local/lib/python3.6/dist-packages/ssm_tools/resolver.py", line 42, in get_list
assert content["ResourceType"] == "EC2Instance"

not working in windows os

Please help me know if anything i need to do to fix this.

C:\Users\Administrator>ec2-session --list --region ap-south-1 --profile vtest
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python311\Scripts\ec2-session.exe\__main__.py", line 4, in <module>
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python311\Lib\site-packages\ssm_tools\ssm_session_cli.py", line 30, in <module>
    signal.signal(signal.SIGTSTP, signal.SIG_IGN)  # Ignore Ctrl-Z - pass it on to the shell
                  ^^^^^^^^^^^^^^
AttributeError: module 'signal' has no attribute 'SIGTSTP'

Argument parsing is incorrect and causes rsync not to work

SSH can accept multiple arguments for a command. rsync relies on this behaviour when you use a command as the remote shell (-e). It should be trivial to use ssm-ssh for that with rsync -e "ssm-ssh --region blah --profile blah, but instead it fails with No instance-id found for destination ..

With some debugging I can see the argument received by ssm-ssh are:

['/home/dqm/.local/bin/ssm-ssh', '--region', 'eu-west-1', '--', '-l', 'root', 'i-0e5169d18eee8506f', 'sudo rsync', '--server', '--sender', '-e.LsfxCIvu', '.', 'my-file']

The failure is caused by ssm-ssh erroneously assuming there is exactly one command argument, and hence the destination must be the second-to-last argument. This is not a valid assumption.

Changing the whole argument parsing is probably not a viable option, but my problem can be solved if a -- argument is respected and any further arguments are passed directly to SSH - the destination would then be guaranteed to be the only freestanding argument before the --.

Listing fails on missing InstanceName and/or Address

When attempting to list available instances by running "ssm-session --list" I receive two KeyErrors intermittently on InstanceName and HostName:

Traceback (most recent call last):
File "/home/srepetski/.local/bin/ssm-session", line 8, in
sys.exit(main())
File "/home/srepetski/.local/lib/python3.6/site-packages/ssm_tools/ssm_session_cli.py", line 81, in main
InstanceResolver(args).print_list()
File "/home/srepetski/.local/lib/python3.6/site-packages/ssm_tools/resolver.py", line 107, in print_list
print(f"** Bad InstanceName is {item['InstanceName']}")
KeyError: 'InstanceName'

When I add some debugging to resolver.py to print the list item when there there are less than 4 elements, I can see that I am getting at least one item with just two elements (InstanceId and HostName), and InstanceName and Address are missing. I'm not sure if the issue is in the data SSM is returning or how the tool is parsing,

srepetski@:~$ ssm-session --list
{'InstanceId': 'i-05a6746e9bf2eeaea', 'HostName': 'uc1bplis536trau'}

Provide a Docker image with ssm-tools (example provided)

It would be awesome to be able to run ssm-tools in a docker image for us on OSX and those that prefer to keep tools off their system. This is what I tried, and it runs ssm-session:

FROM ubuntu:bionic

RUN apt-get update -qq && \
    apt-get install -y python3-pip

RUN pip3 install awscli --upgrade
# See https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-install-plugin.html#install-plugin-debian
ADD https://s3.amazonaws.com/session-manager-downloads/plugin/latest/ubuntu_64bit/session-manager-plugin.deb session-manager-plugin.deb
RUN dpkg -i session-manager-plugin.deb
RUN pip3 install aws-ssm-tools

# ssm-tunnel prerequisities:
RUN apt-get update -qq && \
    apt-get install -y sudo iproute2

ENTRYPOINT ["/bin/bash"]

however ssm-tunnel <instance> fails without any clear indication of why:

root@eb4429e01043:/# ssm-tunnel --verbose --debug i-xxxxxx
[ssm-tunnel] DEBUG: Logging level set to DEBUG
[ssm-tunnel] DEBUG: Spawning: aws  ssm start-session --target i-xxxxxx
[ssm-tunnel] DEBUG: PID: 241
[ssm-tunnel] DEBUG: Starting session with SessionId: john.snow@43213109573-019fead4f60ee2455

[ssm-tunnel] DEBUG: Creating tunnel
[ssm-tunnel] DEBUG: # Agent device tunSSM.122.45 is ready
[ssm-tunnel] DEBUG: command: sudo ip tuntap add tunSSM.122.44 mode tun user 0
open: No such file or directory
[ssm-tunnel] DEBUG: command: sudo ip link set tunSSM.122.44 down
Cannot find device "tunSSM.122.44"
[ssm-tunnel] DEBUG: command: sudo ip tuntap del tunSSM.122.44 mode tun
open: No such file or directory
[ssm-tunnel] INFO: Closing tunnel, please wait...
[ssm-tunnel] DEBUG: Closing session

ssm-tunnel does not find comman ip

When running ssm-tunnelfrom my MacBook I get the following error:

$ ssm-tunnel -v i-123456789 -r 10.0.0.0/8
sudo: ip: command not found
sudo: ip: command not found
sudo: ip: command not found
[ssm-tunnel] INFO: Closing tunnel, please wait...

Is this supposed to work on macOS? The agent is installed on the target Amzn2 instance.

Can't login to ECS task

I wanted to use the new ecs-session to login to our app container but can't figure out how. Our app (kicker) is deployed with a nginx sidecar conainer in each instance.

~> ecs-session --list
dev-Cluster-mWkJ7Y4  service:kicker-service-09C2qGCVy  11f372449e0eda484b486e9b94cc9f74  kicker   172.31.21.13
dev-Cluster-mWkJ7Y4  service:kicker-service-09C2qGCVy  4b5f60b18d6cb83f9f4e7e8e30edf99e  kicker   172.31.39.74
dev-Cluster-mWkJ7Y4  service:kicker-service-09C2qGCVy  11f372449e0eda484b486e9b94cc9f74  nginx   172.31.21.13
dev-Cluster-mWkJ7Y4  service:kicker-service-09C2qGCVy  4b5f60b18d6cb83f9f4e7e8e30edf99e  nginx   172.31.39.74

Now ecs-session won't let me specify both the container and the task id and without that it always finds more than one container and errors.

~> ecs-session kicker
Found 4 instances for: kicker
Use Container IP or Task ID to connect to a specific one
dev-Cluster-mWkJ7Y4  service:kicker-service-09C2qGCVy  11f372449e0eda484b486e9b94cc9f74  kicker   172.31.21.13
dev-Cluster-mWkJ7Y4  service:kicker-service-09C2qGCVy  4b5f60b18d6cb83f9f4e7e8e30edf99e  kicker   172.31.39.74

Same when I try the task id or the ip. How can I login to one of them?

BTW Why does it say "Found 4 instances" when there are only 2?

ssm-session can list but cannot resolve Instance ID of Managed Instances

$ ssm-session --list
mi-abcdef1234567890a   mycomputer.WORKGROUP       10.0.0.16
$ ssm-session mi-abcdef1234567890a
[ssm-session] WARNING: Could not resolve Instance ID for 'mi-abcdef1234567890a'
[ssm-session] WARNING: Perhaps the 'mi-abcdef1234567890a' is not registered in SSM?
$ aws ssm start-session --target mi-abcdef1234567890a

Starting session with SessionId: user-0724d37010e406ad6
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

Try the new cross-platform PowerShell https://aka.ms/pscore6

PS C:\Windows\system32>
# 
$ aws ssm get-inventory --query "Entities[?Id=='mi-abcdef1234567890a']"
[
    {
        "Id": "mi-abcdef1234567890a",
        "Data": {
            "AWS:InstanceInformation": {
                "TypeName": "AWS:InstanceInformation",
                "SchemaVersion": "1.0",
                "CaptureTime": "2021-03-18T00:09:21Z",
                "Content": [
                    {
                        "AgentType": "amazon-ssm-agent",
                        "AgentVersion": "3.0.655.0",
                        "ComputerName": "mycomputer.WORKGROUP",
                        "IamRole": "SSMServiceRole",
                        "InstanceId": "mi-abcdef1234567890a",
                        "IpAddress": "10.0.0.16",
                        "PlatformName": "Microsoft Windows 10 Pro",
                        "PlatformType": "Windows",
                        "PlatformVersion": "10.0.19042",
                        "ResourceType": "ManagedInstance"
                    }
                ]
            }
        }
    }
]

$ aws --version
aws-cli/2.1.30 Python/3.8.8 Darwin/19.6.0 exe/x86_64 prompt/off
$ /usr/local/sessionmanagerplugin/bin/session-manager-plugin  --version
1.2.54.0
$ ssm-session --version
ssm-tools/1.3.2

Looks like the regex for checking instance ids doesn't handle "mi-xxxx"

if re.match('^i-[a-f0-9]+$', instance):

should be

if re.match('^m?i-[a-f0-9]+$', instance):

made the change and tested it, have a PR in a sec.

feature request - add support for --reason option

aws ssm start-session supports a --reason option that is used for documentation and change control since it's part of the API call and written to Cloudtrail.

ec2-session doesn't support this option. It would be very helpful to support this.

https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ssm/start-session.html

Getting thousands of warnings about "SSM inventory entity not recognised"

Firstly, I have a pull request coming, I just wanted to create this issues for reference and justification of the pull request.

Steps to reproduce

ssm-session

Expected:

Brief pause while entity name is resolved, then imediately connects.

Actual:

1000's of warnings like the following before evntually connecting

[ssm-session] WARNING: SSM inventory entity not recognised: {'Id': 'i-***************', 'Data': {'AWS:InstanceInformation': {'TypeName': 'AWS:InstanceInformation', 'SchemaVersion': '1.0', 'CaptureTime': '2022-09-09T04:38:50Z', 'Content': [{'InstanceId': 'i-*************', 'InstanceStatus': 'Terminated'}]}}}

Troubleshooting:

I pulled out some of the code and started reproducing. The main thing I found was that it was choking on a KeyError trying to check the ResourceType attribute. A large number of terminated instances don't have a ResourceType attribute when returned from the API.

For example, the following entity only has InstanceStatus and InstanceId attributes

[ssm-session] WARNING: SSM inventory entity not recognised: {'Id': 'i-*************', 'Data': {'AWS:InstanceInformation': {'TypeName': 'AWS:InstanceInformation', 'SchemaVersion': '1.0', 'CaptureTime': '2022-09-09T07:59:39Z', 'Content': [{'InstanceId': 'i-*************', 'InstanceStatus': 'Terminated'}]}}}

We don't care about terminated, stopped or disconnected hosts anyway so not much point logging/warning about them.

Recommendations

Let's utilize AWS's built in filtering so that we only pull down a subset of records that match our criteria.

Currently in my situation we are pulling down 8800 records and then filtering them downs to just 26, and then resolving the 1 instance.

Again, pull request incoming.

Feature Request: Ignore CTRL+Z (signal.SIGSTP)

I'd like to contribute a change to the code-base, but since it affects the default behaviour of the tool I seek some implementation/design guidance.

The desired behaviour is such that pressing Ctrl+Z on an active SSM session should be handled by the server side, and not the client side.

Current behaviour (drops to local shell)

(local) ➜  ~ ssm-session i-abcdefgh

Starting session with SessionId: todor-12345678efgh
(remote) $ sleep 60

[2]  + 26132 suspended  ssm-session i-abcdefgh
(local) ➜  ~

Desired behaviour (drops to remote shell)

➜  ~ ssm-session i-abcdefgh

Starting session with SessionId: todor-12345678efgh
(remote) $ sleep 60
^Z[1] + Stopped                    sleep 60
(remote) $

For my personal use-case the below is sufficient, but it affects the default behaviour of the tool. Is such a change acceptable? If not - where is a better place to implement it?

diff --git a/ssm-session b/ssm-session
index f659c42..172484a 100755
--- a/ssm-session
+++ b/ssm-session
@@ -3,5 +3,8 @@
 import sys
 from ssm_tools.ssm_session_cli import main

+import signal
+signal.signal(signal.SIGTSTP, signal.SIG_IGN)
+
 if __name__ == "__main__":
     sys.exit(main())

KeyError: 'Tags' when an instance without tags is present

Hello,

We figure out that is an instance rans without tags associated to it, it generates the following error:

Traceback (most recent call last):
  File "/home/jynolen/.local/bin/ssm-session", line 8, in <module>
    sys.exit(main())
  File "/home/jynolen/.local/lib/python3.8/site-packages/ssm_tools/ssm_session_cli.py", line 105, in main
    InstanceResolver(args).print_list()
  File "/home/jynolen/.local/lib/python3.8/site-packages/ssm_tools/resolver.py", line 122, in print_list
    items = self.get_list().values()
  File "/home/jynolen/.local/lib/python3.8/site-packages/ssm_tools/resolver.py", line 93, in get_list
    for tag in instance['Tags']:
KeyError: 'Tags'

Error in accounts with high instance churn / SSM Inventory latency

There appears to be some latency between when an instance is terminated and when it's removed from the SSM inventory. As a result, in accounts with a large number of instances and high churn (I have ~1,000 instances in a single region, and have hundreds of them are replaced every day), it's likely that DescribeInstances will fail on one or more instances that are still in the SSM Inventory. In these cases, aws-ssm-tools' method of calling DescribeInstances with the full list of instance IDs in the SSM Inventory will render it useless as it will encounter an error.

The following is a trimmed-down excerpt of the ssm-session debug output from an account that has 724 instances in the SSM Inventory for this region. 649 of those were properly recognized by SSM Inventory as terminated and skipped (DEBUG: Ignoring terminated instance:), but it looks like 4 of them were terminated within a short enough period that SSM doesn't know they're gone, but DescribeInstances errors:

[ssm-session] DEBUG: Logging level set to DEBUG
[ssm-session] DEBUG: Fetching SSM inventory
...
[ssm-session] DEBUG: Added instance: i-02df27b30a98a2bf6: {'InstanceId': 'i-02df27b30a98a2bf6', 'HostName': 'ip-10-230-8-231'}
...
[ssm-session] DEBUG: Added instance: i-06a5cd08dccd6cf94: {'InstanceId': 'i-06a5cd08dccd6cf94', 'HostName': 'ip-10-230-8-70'}
...
[ssm-session] DEBUG: Added instance: i-093177e4964fd4be3: {'InstanceId': 'i-093177e4964fd4be3', 'HostName': 'ip-10-142-0-108'}
...
[ssm-session] DEBUG: Added instance: i-0f4d372b4cba55a1a: {'InstanceId': 'i-0f4d372b4cba55a1a', 'HostName': 'ip-10-230-12-196'}
...
[ssm-session] ERROR: An error occurred (InvalidInstanceID.NotFound) when calling the DescribeInstances operation: The instance IDs 'i-02df27b30a98a2bf6, i-06a5cd08dccd6cf94, i-093177e4964fd4be3, i-0f4d372b4cba55a1a' do not exist

I can think of a few possible fixes for this:

Right now, if the argument to ssm-session is an Instance ID (i- string), we short-circuit InstanceResolver.resolve_instance() and directly return that Instance ID. The SSM Inventory data includes both Instance IDs and HostNames, but we make the full DescribeInstances call any time our argument isn't an Instance ID. In InstanceResolver.get_list() we could check for hostname matches, and if there are any, return those without needing to call DescribeInstances to get the IP addresses. This would both fix this specific bug in all cases except specifying an IP address, and would also avoid potentially-slow DescribeInstances calls in cases where we don't need them.
We could switch from calling DescribeInstances with all of the SSM Inventory instance IDs to using the boto3 EC2.Instance(instance_id) resource for each instance. This would greatly increase the number of DescribeInstances API calls, and likely adversely effect performance, but would allow us to try/except on each instance ID itself.
Since boto3 captures detailed information on the error, and AWS error messages are usually fixed and reliable strings, we could catch the InvalidInstanceID.NotFound error, parse out the Instance IDs from the error message, remove them from the list of instances, and retry the DescribeInstances operation with the updated list of instance IDs.

I'd be happy to open a pull request to implement any of these options, or a combination of them, but I'll leave the choice of which up to you. Since this should probably be an edge case, I'd personally lean towards a combination of options 1 and 3.

(BTW - This is an absolutely wonderful project! Thank you so much! It's completely revolutionized how I work with EC2 instances when I need access to an actual instance, and I'm working hard to let everyone else at my company know about it.)

ec2-session stuck after executing command

ec2-session instance --region us-east-1 -c "\"sudo -u ec2-user -i /bin/bash -c 'touch 1.txt'\""

Starting session with SessionId: myuser-0001eda76e4e3ee7a

and it never returns or exit. Ctrl+C also wont respond

On MacOS monterrey

mludvig / aws-ssm-tools Goto Github PK

aws-ssm-tools's People

Contributors

Stargazers

Watchers

Forkers

aws-ssm-tools's Issues

Current behaviour (drops to local shell)

Desired behaviour (drops to remote shell)

Recommend Projects

Recommend Topics

Recommend Org

Jobs