We do not guarantee constant uid and gid for created local users at the moment. How could we handle this?

This is a great little cloudformation template.
Was wondering how to change it to apply to a bastion host or any particular EC2 host rather than creating an EC2 during the formation script?

Found an issue when removing the user from the IAM group. If I remove the user from the group, it does not ssh in which is great and it works. But if I put him back to the IAM group (running /opt/import_users.sh manually), it can't login.
The fix i did is to remove the user from the system, usedel user1... and remove his directory.

I found out that it jail the user to /sbin/nologin
user1:x:504:505::/home/user1:/sbin/nologin

So this project got a bit messy because of me not merging stuff fast enough. I would like to get this project back on track. So let's discuss how we continue @shinenelson, @mvanbaak, and @dylansmith

What we have:

• Multi Account functionality: #22 already as a feature issue here #21
• Use IAM Groups to manage access: we have quiet a few implementations here
  *#23 Downside, slower because group membership is checked on login, but we don't get into problems with out of date group memberships
  • #19 does not handle changes in the group memberships?
  • @shinenelson import-user-from-groups branch should work similar than

And we also have a big PR #24 which contains basically all of the above functionality.

My suggestions for v1.0:

• merge Multi Account functionality #22 and close #21
• merge #23
• close #19 and #24
• fix issue #27 (fix is part of #24)
• verify that Documentation is up to date
• verify that iam_ssh_policy.json is up to date
• verify that showcase.yaml is up to date
• verify that install.sh is up to date
• use install.sh in showcase.yaml to not duplicate code
• implement a test that ensures that the available features are working as expected

My suggestion for the future v1.x:

• implement a the better management of local users
  • uuid
  • local groups
  • shell etc etc
• improve performance of login procedure

What's your opinion?

On ubuntu instances service is called ssh

service ssh restart

I plan implement your scripts in our cross-account environment so that I can just install the policys and roles using a cloud formation stack in the service catalog.
After that the users should have the option to download the install.sh script from AWS S3 and use this functionallity when they need it.

It would be cool if there will be an uninstalling script for revert all to basic if they dont need it anymore.

from @nicholascowan

#83 breaks functionality on CENTOS7 because it's checking for init.d scripts and doesn't take into account systemd, the restart method should be systemctl restart sshd.

This causes script to throw an error since ssh.service is not found, and would break in any cloudformation scripts.

Hi Michael,

excellent blog post and thanks for sharing this idea including the code!

I was wondering why you're importing all users (and updating them every 10min). Since you're already running a custom script with AuthorizedKeysCommand couldn't you at that point look up if that user exists in IAM and has a public key and only then create it on the fly? (Or will this command fail before ssh even gets to execture this script because the user isn't already there?)
There could be a simple caching mechanism to avoid doing that for every single SSH connection or once the user is created it will not reevaluate the key (and then there could be a cron deleting those keys/user every hour or so forcing the next login after that to reevaluate).

I'm thinking about adopting this idea, but in our use case we have different teams that need access to different sets of servers (with some overlap). I'm thinking of checking the user's groups (with aws iam list-groups-for-user --user-name ...) and only allow access if the current server "type" (just a key that we made up) matches one of the user's groups.

Lastly one more thought: Our AWS accounts use email addresses as usernames, but those will be rejected by the adduser command. So there probably should be some sanitation before creating the user.

Is there a way to get this to not perform the IAM aws cli command if it finds a .ss/authorized_keys entry?

The reason I ask is we use Ansible to build out environments, and currently it is causing intermittent issues with timeouts on ssh, specially if we are running the playbooks against a large number of instances. The playbooks are using the local ec2-user, so it would really be benificial to us, if when it finds a working key, it doesn't go off to IAM.

Regards

B

The showcase that is also used for testing is not working with an Ubuntu AMI. See ubuntu branch

Operating System: Amazon Linux AMI 2017.03
AWS CLI Version: aws-cli/1.11.83 Python/2.7.12 Linux/4.9.32-15.41.amzn1.x86_64 botocore/1.5.46

Followed the README as best as I can:

1. created iam role and instance profile
2. created ec2 with that IAM role
3. added pub key to user
4. cloned project on server
5. ran install.sh and saw new users created in /etc/passwd
6. tried to ssh and got permission denied

I tried to look for log messages but only saw Dec 13 11:24:03 ip-172-31-17-132 aws-ec2-ssh[20134]: Created new user .... and no log in attempt log messages

What am I missing? Any ideas on how I can debug this?

Thanks!

I absolutely love this solution, it's what I've been looking for for literally years.

I am wondering what its limits are. I'm going to be setting it up for along the lines of 5 users or less - not going too nuts. And not on too many servers, either. I think it'll fit there just fine.

But I can imagine if you have, like 100 users or something - it might start to suck there (all those users getting added - could start to fill up your root disk?). Or if you have maybe 1000 servers - having each of them list every single user available in IAM every 10 minutes could start to cause a strain or throttling of your ability to make IAM queries.

Though, some of the nice things - if your connection to IAM were not quite working, 'old' users could still log in. Though they'd still have to be able to fetch the appropriate SSH key, so even if "List IAM Users" call stopped working, the "Get User's SSH Key" one would have to stay working. But also - even if your VPC NAT gateway goes down, you ought to still be able to log in (since it doesn't require external connectivity).

So I was wondering if that's something you'd consider documenting (if you agree with me on my ideas for where the limits might lie). Or if you'd like me to take a swing at that, I'd be happy to help and put together a PR. I might put it as an 'Advantages/Disadvantages' type of thing if it were up to me.

Anyways, regardless of any of that, thank you 1000 times for this. It's the first time I've found a solution to this problem that feels like it actually fits.

If a user is added to a group manually from the *NIX shell, that group is stripped at the next run of the import_iam_users cron. This is not desirable since we cannot claim aws-ec2-ssh to be the ultimate user management utility on the servers.

There are 3 ways to approach this problem :

1. Do not re-sync the groups of users
2. Use all groups from the user while re-syncing
3. Allow users from specific IAM groups to have local groups. This could complicate the problem since we would have to map each IAM group to their local group, etc.

Use-case :
I use local groups to provide restricted sudo access to a subset of users. I cannot give them complete sudo access (especially with NOPASSWD enabled) and I'll also need to restrict the commands each group is able to run with sudo.

If you use /etc/login.defs to set a custom home basedir (instead of /home), the system breaks when it tries to sync.

I have a custom home basedir set of /something/blah and when the useradd call gets invoked, the home directory is correctly created as /something/blah/${username}. But then a later chown gets invoked, and that seems to be hard-coded to /home/${username}.

In my case, I actually wanted the users to live in /home/ so I made a modification to the sync script to do that. I don't know if that's a configurable option you want to enable or not. Seems a little rarely-used, to me. Not a lot of people are gonna want that, I don't think.

But, regardless of that - the bit where it does the chown probably ought to read the users "real" home directory, rather than assuming it can do the hardcoding.

Maybe you'd want to use getent ${username} and parse out the : marks to get their 'real' home? Specifically, I'm seeing this work for me:

getent passwd $username |cut -d ':' -f 6

When installing the rpm package, the cronjobs at boot and every hour to sync the users are not created.

I'm not sure if every SSH user has an IAM account, but we all have pretty well-secured GitHub accounts. How about adding an option to grab groups and users from GitHub, too?

Maybe we need an adapter with implementations for IAM and GitHub and keep the rest of the code universal.

Let's not forget that AWS CLI comes with JMESPath, which I like more than jq, so, GitHub can be used via curl and JMESPath without requiring client libraries or other packages.

Just an idea. I may git this a try if you give me a blessing.

Instead of

SaveUserName="$1"
SaveUserName=${SaveUserName//"+"/".plus."}
SaveUserName=${SaveUserName//"="/".equal."}
SaveUserName=${SaveUserName//","/".comma."}
SaveUserName=${SaveUserName//"@"/".at."}

the username must be changed in the opposite direction:

UnsaveUserName="$1"
UnsaveUserName=${SaveUserName//".plus."/"+"}
UnsaveUserName=${SaveUserName//".equal."/"="}
UnsaveUserName=${SaveUserName//".comma."/","}
UnsaveUserName=${SaveUserName//".at."/"@"}

(reported via email

Consider the following setup:

• one AWS account (lets call it management) which has all IAM users etc
• multiple AWS accounts (test, staging, production) running EC2 instances

Can you provide any pointers in how we should get the users from the management account on instances in the test/staging/production account?

This way we can manage all users in one account, while using them in many. (following AWS best practices)

Hi I found some errors:

should be:
Setup users account -> 3. Put the dev aacount

Also, for multiple accounts, how can one add trust relations for multiple accounts.

Last thing , please add spesific instruction to run the install.sh script with the a cl parameter

I was trying to install the 1.1 RPM from the README, but noticed the cron wasn't included. Poked around the URL and found there's a 1.4 version, which seems to include it. Might wanna update the URL in the README EDIT: I found it under releases just now.

Correct URL:
https://s3-eu-west-1.amazonaws.com/widdix-aws-ec2-ssh-releases-eu-west-1/aws-ec2-ssh-1.4.0-1.el7.centos.noarch.rpm

Fantastic tool btw, very grateful to have it!

Per http://www.sudo.ws/man/1.8.13/sudoers.man.html:

sudo will read each file in /etc/sudoers.d, skipping file names that end in ‘~’ or contain a ‘.’ character to avoid causing problems with package manager or editor temporary/backup files.

This has an unintended side effect for IAM users that follow a first name.lastname naming convention - assuming John Smith has a username of john.smith, import_user.sh creates a file of /etc/sudoers.d/john.smith which is then ignored by sudo, and the user is prompted for a password when sudoing that (generally) doesn't exist.

Ideally, import_user.sh should be modified to remove all non-alphanumeric characters from the names of any files created under /etc/sudoers.d, while retaining the actual IAM username within the sudoers.d file.

The sed substitution command is not setting DONOTSYNC=1 when installing using RPM

A working and standard sed command could be:

sed -i 's/DONOTSYNC=0/DONOTSYNC=1/g' ${RPM_BUILD_ROOT}%{_sysconfdir}/aws-ec2-ssh.conf

We use email addresses as IAM usernames, but these can easily be longer than 32 chars. The current implementation of clean_iam_username makes them even longer.

It would be great if we could provide our custom function that would be invoked instead of clean_iam_username.

The custom function could be provided in /etc/aws-ec2-ssh.conf file, or maybe in some separate file.

Hi,

I'm testing aws-ec2-ssh on CentOS 7 AMI and getting this error from sshd log:
error: AuthorizedKeysCommand /usr/bin/authorized_keys_command.sh xxxxxxxxx failed, status 255.
When I run /usr/bin/authorized_keys_command.sh xxxxxxx manually it just returns public key as expected.
Sync users script works fine, my user created and has sudo access as expected.

Additional information:
OS: CentOS 7 from AWS Marketplace with latest packages
AWS CLI: installed from EPEL aws-cli/1.11.133 Python/2.7.5 Linux/3.10.0-693.2.2.el7.x86_64 botocore/1.6.0

I'm sure many people noticed a very brief IAM outage earlier this week. During the outage, IAM was not responsive and as a result, this script would go and delete all of the local users synced from IAM as IAM did not return a list of users.

I was hoping to discuss what are the options for some fallback behavior in the event of IAM outage or actually just plain network connectivity outage.

I installed aws-ec2-ssh on an ubuntu 16.04 server running on aws. The import_users.sh script imports the users from aws iam and setup the user environment. But after 10 minutes when the cron job runs the script again, it removes all the users imported. I have also noticed that, it doesn't remove local users created by me, as it should be.
I think the problem exist in the following code:
iam_users=$(get_clean_iam_users | sort | uniq)
sudo_users=$(get_clean_sudoers_users | sort | uniq)
local_users=$(get_local_users | sort | uniq)
intersection=$(echo ${local_users} ${iam_users} | tr " " "\n" | sort | uniq -D | uniq)
removed_users=$(echo ${local_users} ${intersection} | tr " " "\n" | sort | uniq -u)

IAM users can have special chars like @ in their names.

allowed characters for IAM usernames are:

alphanumeric, including the following common characters: plus (+), equal (=), comma (,), period (.), at (@), underscore (_), and hyphen (-).

But this is not allowed for Linux users.

the POSIX ("Portable Operating System Interface for Unix") standard (IEEE Standard 1003.1 2008) states:

3.431 User Name
A string that is used to identify a user; see also User Database. To be portable across systems conforming to POSIX.1-2008, the value is composed of characters from the portable filename character set. The character should not be used as the first character of a portable user name.
3.278 Portable Filename Character Set

The set of characters from which portable filenames are constructed.

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
a b c d e f g h i j k l m n o p q r s t u v w x y z
0 1 2 3 4 5 6 7 8 9 . _ -

Characters that are allowed in IAM user names but not in Linux user names:

plus (+), equal (=), comma (,), at (@).

script will exit after execute /usr/bin/pkill which cause users won't delete after added

function delete_local_user() {
# First, make sure no new sessions can be started
/usr/sbin/usermod -L -s /sbin/nologin "${1}"
# ask nicely and give them some time to shutdown
/usr/bin/pkill -15 -u "${1}"
sleep 5
# Dont want to close nicely? DIE!
/usr/bin/pkill -9 -u "${1}"
sleep 1
# Remove account now that all processes for the user are gone
/usr/sbin/userdel -f -r "${1}"
}

Hi,

I would like to add a tool called shellcheck to lint shell scripts before we merge them into master.

At the moment, the tool finds toe following issues:

$ find . -type f -name '*.sh' -exec shellcheck -s bash {} \;

In ./authorized_keys_command.sh line 8:
[ -f /etc/aws-ec2-ssh.conf ] && . /etc/aws-ec2-ssh.conf
                                ^-- SC1091: Not following: /etc/aws-ec2-ssh.conf was not specified as input (see shellcheck -x).


In ./authorized_keys_command.sh line 13:
: ${ASSUMEROLE:=""}
  ^-- SC2086: Double quote to prevent globbing and word splitting.


In ./import_users.sh line 4:
[ -f /etc/aws-ec2-ssh.conf ] && . /etc/aws-ec2-ssh.conf
                                ^-- SC1091: Not following: /etc/aws-ec2-ssh.conf was not specified as input (see shellcheck -x).


In ./import_users.sh line 7:
: ${DONOTSYNC:=0}
  ^-- SC2086: Double quote to prevent globbing and word splitting.


In ./import_users.sh line 17:
: ${IAM_AUTHORIZED_GROUPS:=""}
  ^-- SC2086: Double quote to prevent globbing and word splitting.


In ./import_users.sh line 20:
: ${LOCAL_MARKER_GROUP:="iam-synced-users"}
  ^-- SC2086: Double quote to prevent globbing and word splitting.


In ./import_users.sh line 23:
: ${LOCAL_GROUPS:=""}
  ^-- SC2086: Double quote to prevent globbing and word splitting.


In ./import_users.sh line 29:
: ${SUDOERSGROUP:=""}
  ^-- SC2086: Double quote to prevent globbing and word splitting.


In ./import_users.sh line 34:
: ${SUDOERS_GROUPS:="${SUDOERSGROUP}"}
  ^-- SC2086: Double quote to prevent globbing and word splitting.


In ./import_users.sh line 39:
: ${ASSUMEROLE:=""}
  ^-- SC2086: Double quote to prevent globbing and word splitting.


In ./import_users.sh line 42:
: ${USERADD_PROGRAM:="/usr/sbin/useradd"}
  ^-- SC2086: Double quote to prevent globbing and word splitting.


In ./import_users.sh line 45:
: ${USERADD_ARGS:="--create-home --shell /bin/bash"}
  ^-- SC2086: Double quote to prevent globbing and word splitting.


In ./import_users.sh line 78:
            aws --region $REGION ec2 describe-tags \
                         ^-- SC2086: Double quote to prevent globbing and word splitting.


In ./import_users.sh line 95:
        for group in $(echo ${IAM_AUTHORIZED_GROUPS} | tr "," " "); do
                            ^-- SC2086: Double quote to prevent globbing and word splitting.


In ./import_users.sh line 126:
            aws --region $REGION ec2 describe-tags \
                         ^-- SC2086: Double quote to prevent globbing and word splitting.


In ./import_users.sh line 166:
    if [[ ! "${username}" =~ ^[0-9a-zA-Z\._\-]{1,32}$ ]]
                                           ^-- SC1001: This \- will be a regular '-' in this context.


In ./import_users.sh line 178:
        ${USERADD_PROGRAM} ${USERADD_ARGS} "${username}"
                           ^-- SC2086: Double quote to prevent globbing and word splitting.


In ./import_users.sh line 179:
        /bin/chown -R "${username}:${username}" "$(eval echo ~$username)"
                                                              ^-- SC2086: Double quote to prevent globbing and word splitting.


In ./import_users.sh line 250:
    intersection=$(echo ${local_users} ${iam_users} | tr " " "\n" | sort | uniq -D | uniq)
                        ^-- SC2086: Double quote to prevent globbing and word splitting.
                                       ^-- SC2086: Double quote to prevent globbing and word splitting.


In ./import_users.sh line 251:
    removed_users=$(echo ${local_users} ${intersection} | tr " " "\n" | sort | uniq -u)
                         ^-- SC2086: Double quote to prevent globbing and word splitting.
                                        ^-- SC2086: Double quote to prevent globbing and word splitting.

If there are no doubts, I would like to activate the tool and I would also fix the existing findings.

Im trying to retro fit this for an ubuntu AMI, however when the script runs import_users.sh it fails as a result of the ubuntu user creation process.

This script works fine for Amazon Linux installs.

Is there any updates for an ubuntu AMI?

The install.sh script relies on finding a commented #AuthorizedKeysCommand none and #AuthorizedKeysCommandUser nobody in the /etc/ssh/sshd_config file. On Ubuntu 16.04 LTS (ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-20170414 (ami-f1d7c395)) these commented entries are not present:
$ grep -i AuthorizedKeysCommand /etc/ssh/sshd_config | wc -l
0
$ grep -i AuthorizedKeysUser /etc/ssh/sshd_config | wc -l
0
This means that the sed command:
sed -i 's:#AuthorizedKeysCommand none:AuthorizedKeysCommand /opt/authorized_keys_command.sh:g' /etc/ssh/sshd_config
does not change /etc/ssh/sshd_config.

Would it be possible to check for the existence of the string that the script is looking to substitute and append the string if it is not found?

For a particular implementation of aws-ec2-ssh, I wanted to override the defaults set in /etc/defaults/useradd. I still wanted "regular" invocations of useradd to respect them, mind you, but for users added via this solution, I wanted those users to specifically live somewhere else.

Furthermore, I noticed that the shell when the users are added from this script is hard-coded to /bin/bash. I figure, we ought to default to whatever the default shell is, but perhaps allow a config variable so that we can set a different default for people added via this script?

I'm thinking some kind of config variable called USERADD_OPTIONS, and you can add things like -b /my/custom/basedir (for my particular problem when I was using a funny homedir), or maybe --shell /bin/tcsh (for those who might want a non-default shell for their aws-ec2-ssh-controlled users).

(If you wanted both, I would think you'd want something like USERADD_OPTIONS="-b /my/custom/basedir --shell /usr/bin/zsh")

My suggestion for the future v1.x:

implement a the better management of local users

• uuid (see #46)
• local groups (implemented by LOCAL_GROUPS option)
• shell etc etc (implemented by #39 )
• Make a single script which can handle both syncing the users as well as getting the key. @mvanbaak here #28 (comment)

[root@ip-10-10-100-4 cron.d]# aws ec2 describe-tags \

        --filters "Name=resource-id,Values=$INSTANCE_ID" "Name=key,Values=$IAM_AUTHORIZED_GROUPS_TAG" \
        --query "Tags[0].Value" \
        --output text \

You must specify a region. You can also configure your region by running "aws configure".

Instead of requiring git and having to git clone (i.e. rely on GitHub, which is often DDoS'd), it would be best if you publish this to PyPI so that we can install it via pip install aws-ec2-ssh.

Hi guys,

I'm wondering if there is a place to add here also creation of .google_authenticator file during user creation process - the content of the file could be the parameter from AWS Parameter Store (encrypted string option).

The Role you are assuming needs only default KMS decrypt and parameter read permission.

Thanks for answers.

First, the current variable SUDOERSGROUP has inconsistent naming (along with ASSUMEROLE) - words are not delimited by an _.

Second, there are use cases in which we want more than a single group having sudo access.

Deprecate SUDOERSGROUP for SUDOERS_GROUPS and, until removed, SUDOERS_GROUPS can default to SUDOERSGROUP.

How would this control sudo access for non-admin users ?

Operating System: Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial
AWS CLI Version: aws-cli/1.14.13 Python/3.5.2 Linux/4.4.0-1041-aws botocore/1.8.17

while running the script install.sh

root@ip-172-31-88-130:/aws-ec2-ssh# sh install.sh
Cloning into 'aws-ec2-ssh'...
remote: Counting objects: 350, done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 350 (delta 1), reused 4 (delta 1), pack-reused 342
Receiving objects: 100% (350/350), 166.58 KiB | 0 bytes/s, done.
Resolving deltas: 100% (177/177), done.
Checking connectivity... done.
install.sh: 161: install.sh: [[: not found
install.sh: 181: install.sh: Syntax error: word unexpected (expecting ")")
root@ip-172-31-88-130:/aws-ec2-ssh#

========================================================

Operating System: Ubuntu
AWS CLI Version: aws-cli/1.11.13 Python/3.5.2 Linux/4.4.0-1044-aws botocore/1.4.70

CloudFormation template, when using Ubuntu, does not reload sshd using the systemd service restart.

Manual installation using install.sh in an interactive console session works as expected on Ubuntu.

I'm trying to get the multi-account setup working according to the example on the README. Whenever I run the install script on an instance in the dev account, I get this:

module.iam-ssh-install.aws_instance.example (remote-exec): An error occurred (IncompleteSignature) when calling the GetGroup operation: Credential must have exactly 5 slash-delimited elements, e.g. keyid/date/region/service/term, got '<key removed>/20171023/us-east-1/iam/aws4_request'

The key has slashes in it. Does the script need to escape the value? Any ideas on how to fix this?

Operating System: Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial
AWS CLI Version: aws-cli/1.14.13 Python/3.5.2 Linux/4.4.0-1041-aws botocore/1.8.17
Error

when i run the CF template i get the following error

Hello,
I have been using this script for a few days and notices that users with names that contain numbers get /sbin/nologin and get added to removed users.

I am wondering if this might be the issue: https://github.com/widdix/aws-ec2-ssh/blob/master/import_users.sh#L164-L169

    iam_users=$(get_iam_users | sort | uniq)
    sudo_users=$(get_sudoers_users | sort | uniq)
    local_users=$(get_local_users | sort | uniq)

    intersection=$(echo ${local_users} ${iam_users} | tr " " "\n" | sort | uniq -D | uniq)
    removed_users=$(echo ${local_users} ${intersection} | tr " " "\n" | sort | uniq -u)

local_users have the local format I.E: robert.at.gmail.com
while iam_users have the normal email format. I.E: [email protected]

Thank you

Right now, if you want to use the ./install.sh script, you need to directly edit it first. It's not a particularly big deal, but I bet it wouldn't be too hard to instead make it use environment variables instead?

For instead, you'd say something like:

IAM_AUTHORIZED_GROUPS=admins,devops SUDOERSGROUP=admins ./install.sh

Just means you can execute the script as-is, in-place, without editing it.

If this is something you'd be interested in, I'd be happy to put together a quick PR for it.

see #38 for discussion

Hi,

I have updated the cronjob with the full path that i have got after executing "which aws" but still after cron job gets executed, I have seen this error

/opt/import_users.sh: line 63: aws: command not found

Can you help me please

Would be great if this created a local log for easy tracking/debugging of users being added/removed etc.
I see it had info logging before. I suggest rather than log on every run, only log when taking important actions like creating a new user or removing a user and so on. Everytime the cron runs and nothing has changed it wouldn't add to the log. I know some of this will be logged in standard system logs already but a log for actions taken by this plugin would still be very useful.