- Save money by stopping EC2 instances and RDS databases during off-hours
- Take backups more frequently
- Use tags to schedule operations
- Secure tags and backups using Identity and Access Management (IAM) policies
- Install and update easily, with CloudFormation (and optionally, StackSets)
Jump to: Installation • Operation Tags • Schedule Tags • Logging • Security • Multi-region/multi-account
In July, 2018, Amazon introduced Data Lifecycle Manager. It's a start, but...
- Tags determine which volumes will be backed up, not when. There is a new, single-purpose API for scheduling.
- Snapshots are taken only every 12 or 24 hours.
- "Last x" snapshot retention is not flexible enough for true archival policies (e.g., keep daily snapshots for a month, plus monthly snapshots for a year).
- You can create snapshots of single volumes, but not images covering all of an EC2 instance's volumes. Also missing is an option to reboot first.
- The same IAM role confers authority to create and delete snapshots, and anyone who can update a lifecycle policy can reduce the retention period to delete snapshots. These are significant risks, especially in light of the data loss prevention provisions in the European Union General Data Protection Regulation.
By all means, set up Data Lifecycle Manager if you have no automation in place, but consider this project for more flexibility -- and sign up as a contributor if you would like to help develop the backup retention part!
-
Log in to the AWS Web Console as a privileged user.
Security Tip: To see what you'll be installing, look in the CloudFormation template.
grep 'Type: "AWS::' aws_tag_sched_ops.yaml | sort | uniq
works well. -
Go to Instances in the EC2 Console. Right-click the Name or ID of an instance, select Instance Settings, and then select Add/Edit Tags. Add:
Key Value Note managed-image
Leave value blank managed-image-periodic
d=*,H:M=11:30
Replace 11:30
with current UTC time + 20 minutes -
Go to the S3 Console. Click the name of the bucket where you keep AWS Lambda function source code. (This may be the same bucket where you keep CloudFormation templates.) If you are creating the bucket now, be sure to create it in the region where you intend to install TagSchedOps; appending the region to the bucket name (for example,
my-bucket-us-east-1
) is recommended. Upload the compressed source code of the AWS Lambda function,aws_tag_sched_ops_perform.py.zip
Security Tip: Remove public read and write access from the S3 bucket. Carefully limit write access.
Security Tip: Download the file from S3 and verify it. (In some cases, you can simply compare the ETag reported by S3.)
md5sum aws_tag_sched_ops_perform.py.zip
should matchaws_tag_sched_ops_perform.py.zip.md5.txt
-
Go to the CloudFormation Console. Click Create Stack. Click Choose File, immediately below "Upload a template to Amazon S3", and navigate to your locally downloaded copy of
cloudformation/aws_tag_sched_ops.yaml
. On the next page, set:Item Value Stack name TagSchedOps
LambdaCodeS3Bucket Name of your S3 bucket MainRegion Current region, if other than us-east-1
For all other paramters, keep the default values.
-
After 20 minutes, check Images in the EC2 Console.
-
Before deregistering (deleting) the sample image, note its ID, so that you can delete the associated Snapshots. Also untag the instance.
-
Go to Users in the IAM Console. Click your regular (uprivileged) username. Click Add Permissions, then click "Attach existing policies directly". In the Search box, type
TagSchedOpsAdminister
. Add the two matching policies.Security Tip: Review everyone's EC2 and RDS tagging privileges!
-
Log out of the AWS Console. You can now manage relevant tags, view logs, and decode errors, without logging in as a privileged user.
-
Check that scheduled AWS operations have completed successfully. Verification of completion is beyond the scope of this code.
-
Test your backups! Can they be restored successfully?
-
Weigh the benefits of rebooting against the risks. Rebooting is usually necessary to make software updates take effect, but a system may stop working afterward.
-
Be aware of AWS charges, including but not limited to: the costs of running the AWS Lambda function, storing CloudWatch logs, and storing images and snapshots; the whole-hour cost when you stop an RDS, EC2 Windows, or EC2 commercial Linux instance (but most EC2 instances have a 1-minute minimum charge); the continuing cost of storage for stopped instances; and costs that resume when AWS automatically starts an RDS instance that has been stopped for too many days.
-
Secure your own AWS environment. Test the function and the IAM policies from end-to-end, to make sure that they work correctly and meet your expectations. To help improve this project, please submit bug reports and feature requests, as well as proposed changes.
-
To enable an operation, add a tag from the table. Leave the value blank.
Start Create Image Reboot then Create Image Reboot then Fail Over Reboot Create Snapshot Create Snapshot then Stop Stop EC2 compute instance managed-start
managed-image
managed-reboot-image
managed-reboot
managed-stop
EC2 EBS disk volume managed-snapshot
RDS database instance managed-start
managed-reboot-failover
managed-reboot
managed-snapshot
managed-snapshot-stop
managed-stop
-
Also add tags for repetitive (
-periodic
) and/or one-time (-once
) schedules. Prefix with the operation. -
If there are no corresponding schedule tags, an enabling tag will be ignored, and the operation will never occur.
-
To temporarily suspend an operation, delete its enabling tag. You may leave its schedule tag(s) in place.
-
Examples (for an EC2 or RDS instance):
Set of Tags Works? Comment managed-start
managed-start-periodic
=u=1,H=09,M=05
Yes Enabled and scheduled managed-start
=No
managed-start-periodic
=u=1,H=09,M=05
Yes Value of enabling tag is always ignored managed-start
managed-start-once
=2017-12-31T09:05
Yes managed-start
managed-start-periodic
=u=1,H=09,M=05
managed-start-once
=2017-12-31T09:05
Yes Both repetitive and one-time schedule tags are allowed managed-start
No No schedule tag managed-start-once
=2017-12-31T09:05
No No enabling tag (operation is suspended) managed-start
managed-start-once
No Blank schedule managed-start
managed-start-periodic
=Monday
No Invalid schedule managed-start
managed-stop-periodic
=u=1,H=09,M=05
No Enabling tag and schedule tag cover different operations
- All times are UTC, on a 24-hour clock.
- The function runs once every 10 minutes. The last digit of the minute is always ignored. For example,
M=47
means one time, between 40 and 50 minutes after the hour. - Month and minute values must have two digits. Use a leading zero if necessary. (Weekday numbers have only one digit.)
- Use a comma (
,
) or a space ( T
separates date from time; it is invariable.- Repetitive (
-periodic
) and one-time (-once
) schedule tags are supported. Prefix with the operation. - If the corresponding enabling tag is missing, schedule tags will be ignored, and the operation will never occur.
-
Tag suffix:
-periodic
-
Values: one or more of the following components:
Name Minimum Maximum Wildcard Combines With Day of month ( d
)d=01
d=31
d=*
H
andM
, orH:M
Weekday ( u
)u=1
(Monday)u=7
(Sunday)H
andM
, orH:M
Hour ( H
)H=00
H=23
H=*
d
oru
, andM
Minute ( M
)M=00
M=59
d
oru
, andH
Hour and minute ( H:M
)H:M=00:00
H:M=23:59
d
oru
Day of month, hour and minute ( dTH:M
)dTH:M=01T00:00
dTH:M=31T23:59
Weekday, hour and minute ( uTH:M
)uTH:M=1T00:00
uTH:M=7T23:59
- To be valid, a component or combination of components must specify a day, hour and minute.
- Repeat a whole component to specify multiple values. For example,
d=01,d=11,d=21
means the 1st, 11th and 21st days of the month. - The
*
wildcard is allowed for day (every day of the month) and hour (every hour of the day). - For consistent one-day-a-month scheduling, avoid
d=29
throughd=31
. - Label letters match
strftime
and weekday numbers are ISO 8601-standard (different fromcron
).
-
Examples:
Value of -periodic
Schedule TagDemonstrates Operation Begins d=28,H=14,M=25
ordTH:M=28T14:25
Monthly event Between 14:20 and 14:30 on the 28th day of every month. d=1,d=8,d=15,d=22,H=03,H=19,M=01
cron
-style scheduleBetween 03:00 and 03:10 and again between 19:00 and 19:10, on the 1st, 8th, 15th, and 22nd days of every month. d=*,H=*,M=15,M=45,H:M=08:50
Extra event in the day Between 10 and 20 minutes after the hour and 40 to 50 minutes after the hour, every hour of every day, and also every day between 08:50 and 09:00. d=*,H=11,M=00,uTH:M=2T03:30,uTH:M=5T07:20
Extra weekly events Between 11:00 and 11:10 every day, and also every Tuesday between 03:30 and 03:40 and every Friday between 07:20 and 7:30. u=3,H=22,M=15,dTH:M=01T05:20
Extra monthly event Between 22:10 and 22:20 every Wednesday, and also on the first day of every month between 05:20 and 05:30. Remember to use spaces (
,
) in RDS! (Either separator character is fine in EC2.)
-
Tag suffix:
-once
-
Values: one or more ISO 8601 combined date and time strings, of the form
2017-03-21T22:40
(March 21, 2017, in this example)- Remember, the code runs once every 10 minutes and the last digit of the minute is ignored
- Omit seconds and fractions of seconds
- Omit time zone; UTC is always used
-
After logging in to the AWS Web Console, go to the Log Group for the AWS Lambda function, in the CloudWatch Logs Console. If you gave the CloudFormation stack a name other than
TagSchedOps
, check the list of Log Groups for all AWS Lambda functions instead. -
Sample output:
initiated
rsrc_id
op
child_rsrc_type
child
child_op
note
9
2017-09-12T20:40
1
i-08abefc70375d36e8
reboot-image
Image
zm-my-server-20170912T2040-83xx7
1
i-08abefc70375d36e8
reboot-image
Image
ami-bc9fcbc6
tag
1
i-04d2c0140da5bb13e
start
0
i-09cdea279388d35a2
start,stop
OPS_UNSUPPORTED
0
my-database
reboot-failover
... ForceFailover cannot be specified
...This run began September 12, 2017 between 20:40 and 20:50 UTC. An EC2 instance (ID prefix
i-
) is being rebooted and backed up, but the instance may not yet be ready again, and the image may not yet be complete; the image is namedzm-my-server-20170912T2040-83xx7
. The image has received IDami-bc9fcbc6
, and has been tagged. A different EC2 instance is starting up, but may not yet be ready. A third EC2 instance is tagged for simultaneous start and stop, a combination that is not supported. An RDS database instance (noi-
orvol-
ID prefix) could not be rebooted with fail-over. (The full error message goes on to explain that it is not multi-zone.) -
There is a header line, an information line, and one line for each operation requested. (Tagging is usually a separate operation.)
-
Values are tab-separated (but the CloudWatch Logs Console seems to collapse multiple tabs).
-
Columns and standard values:
initiated
rsrc_id
op
child_rsrc_type
child
child_op
note
Operation initiated? Resource ID Operation Child type Pointer to child Child operation Message 0
No1
Yes9
Info. onlyi-
EC2 instance IDvol-
EBS volume ID
RDS instance nameSee table Image
Snapshot
DBSnapshot
Name, ID, or ARN, as available tag
-
Although the TagSchedOpsAdminister and TagSchedOpsTagSchedule policies authorize read-only access to the logs via the AWS API, and seem to be sufficient for using the links provided above, users who are not AWS administrators may also want additional privileges for the CloudWatch Console.
If the DEBUG
environment variable is set, the function outputs internal reference data, including the regular expressions used to match schedule tags.
To use the debugging mode,
-
Log in to the AWS Web Console as a privileged user. AWS Lambda treats changes to environment variables like changes to code.
-
Click on the TagSchedOpsPerformLambdaFn AWS Lambda function.
-
Open the Code tab and scroll to the bottom. In the "Environment variables" section, type
DEBUG
in the first empty Key box. Leave Value blank. -
Scroll back to the top and click the white Save button. Do not click the orange "Save and test" button; that would cause the function to run more than once in the same 10-minute interval.
-
After 10 minutes, find the debugging information in CloudWatch Logs.
-
Turn off debugging mode right away, because the extra information is lengthy. Back on the Code tab, scroll down and click Remove, to the far right of
DEBUG
. Repeat Step 4 to save.
-
The TagSchedOpsAdminister policies authorize turning the function on or off completely.
-
After logging in to the AWS Web Console, go to Rules in the CloudWatch Events Console. Click the radio button to the left of TagSchedOpsPerform10MinEventRule, then select Enable or Disable from the Actions pop-up menu, next to the blue "Create rule" button.
-
This toggle is per-region and per-AWS-account.
-
Missed operations will not occur when the function is turned back on; there is no queue or backlog.
Some operations create a child resource (image or snapshot) from a parent resource (instance or volume).
-
The name of the child consists of these parts, separated by hyphens (
-
):# Part Example Purpose 1 Prefix zm
Identifies and groups children created by this project, for interfaces that do not expose tags. z
will sort after most manually-created images and snapshots.m
stands for "managed".2 Parent name or identifier webserver
Conveniently indicates the parent. Derived from the Name
tag (if not blank), the logical name (if supported), or the physical identifier (as a last resort). Multiple children of the same parent will sort together, by creation date (see next row).3 Date/time 20171231T1400Z
Indicates when the child was created. Always includes the Z
suffix to indicate UTC. The last digit of the minute is normalized to 0. The-
and:
separators are removed for brevity, and because AWS does not allow:
in resource names. (Themanaged-date-time
tag preserves the separators.)4 Random string g3a8a
Guarantees unique names. Five characters are chosen from a small set of unambiguous letters and numbers. -
If parsing is ever necessary, keep in mind that the second part may contain internal hyphens.
-
This project replaces characters forbidden by AWS with
X
. -
For some resource types, the description is also set to the name, in case interfaces expose only one or the other.
-
Special tags are added to the child:
Tag(s) Purpose Name
Supplements EC2 resource identifier. The key is renamed managed-parent-name
when the value is passed from parent to child, because the child has aName
tag of its own. The code handlesName
specially for both EC2 and RDS, in case AWS someday extends EC2-style tag semantics to RDS.managed-parent-name
The Name
tag value from the parent. May be blank.managed-parent-id
The identifier of the parent instance or volume. AWS stores this in metadata for some but not all resource types, and the retrieval key differs for each resource type. managed-origin
The operation (for example, snapshot
) that created the child. Identifies resources created by this project. Also distinguishes special cases, such as whether an EC2 instance was or was not rebooted before an image was created.managed-date-time
Groups resources created during the same 10-minute interval. The last digit of the minute is normalized to 0, and Z
is always appended, to indicate UTC. AWS stores the exact time (too specific for grouping) in metadata, and the retrieval key and the format differ for each resource type! -
Tags other than operation-enabling tags, schedule tags, and the
Name
tag, are copied from parent to child. (The deletion tag,managed-delete
, would not make sense on instances and volumes, but if it is present, it is not copied to images and snapshots.)
Multiple non-simultaneous operations on the same resource are allowed.
If two or more operations on the same resource are scheduled for the same 10-minute interval, the function combines them, where possible:
Resource | Simultaneous Operations | Effect |
---|---|---|
EC2 instance | Create Image + Reboot | Reboot then Create Image |
EC2 or RDS instance | Stop + Reboot | Stop |
RDS instance | Stop + Create Snapshot | Create Snapshot then Stop |
The Create Image + Reboot combination for EC2 instances is useful. For example, you could take hourly backups but reboot only in conjunction with the midnight backup. The midnight backup would be guaranteed to be coherent for all files, but you could safely retrieve static files as of any given hour, from the other backups. To set up this example:
Tag | Value |
---|---|
managed-image |
|
managed-image-periodic |
d=*,H=*,M=59 |
managed-reboot |
|
managed-reboot-periodic |
d=*,H=23,M=59 |
23:59, which for the purposes of this project represents the last 10-minute interval of the day, is the unambiguous way to express almost the end of some designated day, on any system. 00:00 and 24:00 could refer to the start or the end of the designated day, and not all systems accept 24:00, in any case. Remember that all times are UTC; adjust for night-time in your time zone!
Resources tagged for unsupported combinations of operations are logged (with message OPS_UNSUPPORTED
) and skipped.
Bad Combination | Reason | Example |
---|---|---|
Mutually exclusive operations | The operations conflict. | Start + Stop |
Choice of operation depends on current state of instance | The state of the instance could change between the status query and the operation request. | Start + Reboot |
Sequential or dependent operations | The logical order cannot always be inferred. Also, operations proceed asynchronously; one might not complete in time for another to begin. Note that Reboot then Create Image (EC2 instance) and Create Snapshot then Stop (RDS instance) are single AWS operations. | Start + Create Image |
-
Prevent unauthorized changes to the AWS Lambda function by attaching the TagSchedOpsPerformLambdaFnProtect IAM policy to most entities with write privileges for:
- AWS Lambda
- CloudFormation Events
- CloudFormation Logs
- IAM (roles and/or policies)
-
Allow only a few trusted users to tag EC2 and RDS resources, because tags determine which resources are started, backed up, rebooted, and stopped.
-
Tag backups for deletion, but let a special IAM user or role actually delete them. To mark images and snapshots for (manual) deletion, add the
managed-delete
tag. -
Do not allow the same entities that create backups to delete backups (or even to tag them for deletion).
-
Choose from a library of IAM policies:
Policy Name Manage Enabling Tags Manage One-Time Schedule Tags Manage Repetitive Schedule Tags Back Up Manage Deletion Tag Delete Scope → Instances, Volumes Instances, Volumes Instances, Volumes Instances, Volumes Images, Snapshots Images, Snapshots TagSchedOpsAdminister Allow Allow Allow No effect Deny Deny TagSchedOpsTagScheduleOnce Deny i Allow ii Deny No effect Deny Deny TagSchedOpsTagSchedulePeriodic Deny i No effect Allow ii No Effect Deny Deny TagSchedOpsTagForDeletion Deny Deny Deny Deny Allow Deny TagSchedOpsBackupDelete Deny Deny Deny Deny Deny Allow TagSchedOpsNoTag Deny Deny Deny No effect Deny Deny Footnotes:
- For RDS, No Effect.
- Enabling tag required. For example, a user could only add
managed-image-once
to an EC2 instance already tagged withmanaged-image
.
These policies cover all regions. If you use regions to differentiate production and non-production resources, copy the policies and adapt them.
Because Deny always takes precendence in IAM, some policy combinations conflict.
A known shortcoming is that, in some cases, you cannot add, change or delete more than one tag in the same operation.
-
Although the TagSchedOpsAdminister and TagSchedOpsTag policies authorize tagging via the AWS API, users who are not AWS administrators may also want:
- AmazonEC2ReadOnlyAccess, to use the EC2 Console
- AmazonRDSReadOnlyAccess, to use the RDS Console
-
You may have to decode authorization errors. The TagSchedOpsAdminister and TagSchedOpsTag policies grant the necessary privilege.
-
Note these AWS technical limitations/oversights:
-
An IAM user or role that can create an image of an EC2 instance can force a reboot by omitting the
NoReboot
option. (Explicitly denying the reboot privilege does not help.) The unavoidable pairing of a harmless privilege, taking a backup, with a risky one, rebooting, is unfortunate. -
Tags are ignored when deleting EC2 images and snapshots. Limit EC2 image and snapshot deletion privileges -- even Ec2TagSchedOpsDelete -- to highly-trusted entities.
-
In RDS, an IAM user or role that can add specific tags can add any other tags in the same request. Limit RDS tagging privileges -- even the provided policies -- to highly-trusted users.
-
Before starting a multi-region and/or multi-account installation, check all regions, in all AWS accounts, for TagSchedOps CloudFormation stacks created using the Quick Start instructions. Delete them now.
If you intend to install TagSchedOps in multiple regions,
-
Create S3 buckets in all regions where you intend to install TagSchedOps. The bucket names must share the same prefix, followed by a hyphen (
-
) and a suffix for the region. The region in which each bucket is created must match the suffix in the bucket's name. -
Upload
aws_tag_sched_ops_perform.py.zip
to each bucket. The need for copies in multiple regions is an AWS Lambda limitation. -
Keep the following rules in mind when setting parameters, later:
Parameter Value LambdaCodeS3Bucket Use the shared prefix; for example, if you created my-bucket-us-east-1
andmy-bucket-us-west-2
, usemy-bucket
MainRegion Always use the same value, to prevent the creation of duplicate sets of user policies StackSetsOrMultiRegion Yes TagSchedOpsPerformCodeS3VersionID Leave blank, because the value would differ in every region; only the latest version of the AWS Lambda function source code file in each region's S3 bucket can be used
If you intend to install TagSchedOps in multiple AWS accounts,
-
In every target AWS account, create
cloudformation/tag-sched-ops-install.yaml
. Set:Item Value Stack name TagSchedOpsInstall
AWSCloudFormationStackSetExecutionRoleStatus Choose carefully! AdministratorAccountId AWS account number of main (or only) account; leave blank if AWSCloudFormationStackSetExecutionRole existed before this stack was created LambdaCodeS3Bucket Name of AWS Lambda function source code bucket (shared prefix, in a multi-region scenario) -
For the AWS Lambda function source code S3 bucket in each region, create a bucket policy allowing access by every target AWS account's AWSCloudFormationStackSetExecutionRole (StackSet installation) or TagSchedOpsCloudFormation role (manual installation with ordinary CloudFormation). The full name of the TagSchedOpsCloudFormation role will vary; for every target AWS account, look up the random suffix in the list of IAM roles or by selecting the TagSchedOpsInstall stack in the list of CloudFormation stacks and drilling down to Resources. S3 bucket policy template:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": [ "arn:aws:iam::TARGET_AWS_ACCOUNT_NUMBER_1:role/AWSCloudFormationStackSetExecutionRole", "arn:aws:iam::TARGET_AWS_ACCOUNT_NUMBER_1:role/TagSchedOpsInstall-TagSchedOpsCloudFormation-RANDOM_SUFFIX_1", "arn:aws:iam::TARGET_AWS_ACCOUNT_NUMBER_2:role/AWSCloudFormationStackSetExecutionRole", "arn:aws:iam::TARGET_AWS_ACCOUNT_NUMBER_2:role/TagSchedOpsInstall-TagSchedOpsCloudFormation-RANDOM_SUFFIX_2" ] }, "Action": [ "s3:GetObject", "s3:GetObjectVersion" ], "Resource": "arn:aws:s3:::BUCKET_NAME/*" } ] }
-
Follow the multi-region steps, even for a multi-account, single-region scenario.
-
Follow the multi-account steps. In a single-account, multi-region scenario, no S3 bucket policy is needed.
-
If StackSets has never been used, create AWSCloudFormationStackSetAdministrationRole. Do this one time, in your main (multi-account scenario) or only (single-account scenario) AWS account. There is no need to create AWSCloudFormationStackSetExecutionRole using Amazon's template; the TagSchedOpsInstall stack provides it, when necessary.
-
In the AWS account with the AWSCloudFormationStackSetAdministrationRole, go to the StackSets Console.
-
Click Create StackSet, then select "Upload a template to Amazon S3", then click Browse and select your locally downloaded copy of
cloudformation/aws_tag_sched_ops.yaml
. On the next page, set:Item Value StackSet name TagSchedOps
LambdaCodeS3Bucket Use the shared prefix; for example, if you created my-bucket-us-east-1
, use usemy-bucket
MainRegion Must be a StackSet target region StackSetsOrMultiRegion Yes TagSchedOpsPerformCodeS3VersionID In a multi-region scenario, leave blank -
On the next page, specify the target AWS accounts, typically by entering account numbers below "Deploy stacks in accounts". Then, move the target region(s) from "Available regions" to "Deployment order". It is a good idea to put the main region first.
Manual installation is adequate if the number of installations is small, but keeping more than one installation up-to-date could be difficult.
-
Follow the multi-region steps, if applicable.
-
Follow the multi-account steps, if applicable.
-
Follow the Quick Start installation steps in each target region and/or target AWS account. Set parameters based on the multi-region and/or multi-account rules.
-
In a multi-account scenario, select the TagSchedOpsCloudFormation IAM Role during this step; CloudFormation may not be able to create the stack without assuming this role.
If the user invoking CloudFormation is not an administrator, attach the following policies to the user beforhand:
Policy Source AmazonS3FullAccess AWS AmazonSNSReadOnlyAccess AWS IAMReadOnlyAccess AWS TagSchedOpsCloudFormationRolePass TagSchedOpsInstall stack CloudFormationFullAccess TagSchedOpsInstall stack Detach these policies -- particularly AmazonS3FullAccess and CloudFormationFullAccess -- after the stack has been created. AWS does not publish the minimum privileges needed to create a stack. Full S3 access is obviously risky, and full CloudFormation access allows a non-administrator to modify or delete any stack with an IAM Role.
-
New versions of the AWS Lambda function source code and the CloudFormation template will be released from time to time.
-
Log in to the AWS Web Console as a privileged user.
-
Go to the S3 Console. Click the name of the bucket where you keep CloudFormation templates and their dependencies. Open the Properties tab. If Versioning is disabled, click anywhere inside the box, select "Enable versioning", and click Save.
-
Open the Overview tab. Upload the latest version of
aws_tag_sched_ops_perform.py.zip
to S3. -
Click the checkbox to the left of the newly-uploaded file. In the window that pops up, look below the Download button and reselect "Latest version". In the Overview section of the pop-up window, find the Link and copy the text after
versionId=
. (The Version ID will not appear unless you expressly select "Latest version".)Security Tip: Download the file from S3 and verify it. (In some cases, you can simply compare the ETag reported by S3.)
md5sum aws_tag_sched_ops_perform.py.zip
should matchaws_tag_sched_ops_perform.py.zip.md5.txt
-
Go to Stacks in the CloudFormation Console. Click the checkbox to the left of
TagSchedOps
(you might have given the stack a different name). From the Actions pop-up menu next to the blue Create Stack button, select Create Change Set For Current Stack. -
Click Choose File, immediately below "Upload a template to Amazon S3", and navigate to your locally downloaded copy of the latest version of
cloudformation/aws_tag_sched_ops.yaml
. On the next page, set:Item Value Change set name Type a name of your choice TagSchedOpsPerformCodeS3VersionID Paste the Version ID, from S3 A different S3 Version ID makes CloudFormation recognize new AWS Lambda function source code. Once you are familiar with the full update procedure, you may skip Steps 2-5 and leave TagSchedOpsPerformCodeS3VersionID as it was, when you are certain that only the CloudFormation template, not the function source code, is changing.
-
Click through the remaining steps. Finally, click "Create change set".
-
In the Changes section, check the Replacement column.
If True is shown on any row, check the Logical ID.
-
If the resource is for internal use, ignore it.
-
If, however, it a user policy, such as TagSchedOpsAdminister, open another Web browser window, go to Policies in the IAM Console, click the name of the policy, open the "Attached entities" tab, and detach the policy from all entities. Keep notes!
-
-
Click Execute, below the top-right corner of the CloudFormation Console window.
-
Refresh until the stack's status shows
UPDATE_COMPLETE
, in green. -
If you had to detach any IAM policies, return to the IAM Console and attach the replacement policies to the original entities.
-
If TagSchedOps is installed in multiple regions, repeat the update steps in each region. The S3 Version IDs will differ.
Differences when updating a StackSet instead of an ordinary stack:
-
Click the radio button to the left of TagSchedOps, in the list of StackSets. From the Actions pop-up menu next to the blue Create StackSet button, select "Manage stacks in StackSet". Then, select "Edit stacks". On the next page, select "Upload a template to Amazon S3" and upload the latest version of
cloudformation/aws_tag_sched_ops.yaml
. -
A single update covers all target regions and/or AWS target accounts.
-
The TagSchedOpsPerformCodeS3VersionID parameter must remain blank. So that CloudFormation will recognize new source code for the AWS Lambda function, rename
aws_tag_sched_ops_perform.py.zip
toaws_tag_sched_ops_perform_20170924.py.zip
(substitute current date) before uploading the file to the regional S3 bucket(s). Change the TagSchedOpsPerformCodeName parameter accordingly. -
Change Sets are not supported. There is no preliminary feedback about the scope of changes.
-
Automated testing, consisting of a CloudFormation template to create sample AWS resources, and a program (perhaps another AWS Lambda function!) to check whether the intended operations were performed. An AWS Lambda function would also be ideal for testing security policies, while cycling through different IAM roles.
-
Archival policy system (ISO 8601 standard duration/interval syntax will be used), and automatic application of
managed-delete
to expired backups. A correct archival policy is not strictly age-based. For example, you might preserve the last 30 daily backups, and beyond 30 days, the first backup of every month. Consider the flaw in the snapshot retention property of RDS database instances: the daily automatic snapshots created when that property is set can never be kept longer than 35 days. -
Additional AWS Lambda function, to automatically delete backups tagged
managed-delete
-
Makefile
-
Tags and reference dictionary updates to support scheduled restoration of images and snapshots (for backup testing?)
This work is dedicated to Ernie Salazar, Régis and Marianne Marcelin, and my wonderful colleagues of the past few years.
Scope | License | Copy Included |
---|---|---|
Source code files | GNU General Public License (GPL) 3.0 | zlicense-code.txt |
Source code within documentation files | GNU General Public License (GPL) 3.0 | zlicense-code.txt |
Documentation files (including this readme file) | GNU Free Documentation License (FDL) 1.3 | zlicense-doc.txt |
Copyright 2018, Paul Marcelin
Contact: marcelin
at cmu.edu
(replace at with @
)