Comments (19)
Here's how I've since been dealing with the problem. I have a top level directory with all templates and a directory for each environment:
|-- stack_master/
+-- templates/
-- web.yml
+-- production/
-- stack_master.yml
+-- parameters/
+-- us-east-1/
-- web.yml
+-- staging/
-- stack_master.yml
+-- parameters
+-- us-east-1/
-- web.yml
This solves two problems:
- I get to define my staging and production environments in the same region
- I don't use the same stack master config (which is 400 lines at the moment) for everything, it becomes more manageable.
I plan on having a seperate folder per AWS account actually, but I like this solution because it doesn't force you into an abstraction, you get to chose whatever you want to.
from stack_master.
I just added support for profiles in 6627f5a
from stack_master.
This is being worked on in #115 but it hasn't had <3 in a while. I too would like this feature, but it's not a high enough priority for me to work on at the moment.
from stack_master.
@patrobinson Thanks for pointing that out ... I have forked the repo and started to examine stack_master ... the work got started on the feature but I somehow lost track what is still missing. To me it looks like that load_stacks is already wrongly behaving and/or was maybe the point were development stopped. Will try to do something about it ... not confident enough, yet.
from stack_master.
I have almost understood the logic there ... but its a bit strange, is it by design that the region_alias will get replaced by the region in the config ... thats also the reason why this is not working yet, the alias is not considered anymore and will simply get replaced without any further use ... which is an approach I understand considering the alias is a real alias and not more ... maybe this should be addressed differently
from stack_master.
Would something like this be in line with stack_master ? (it would behave as usual but if the alias itself does hold a hash it affects the way how parameters are getting resolved ... this would address my requirement, I am not 100% sure about the original requirement of adding a prefix to the stack-name)
region_aliases:
testing:
region: us-east-1
explicit_alias_parameters: true
production: us-east-1
staging: ap-southeast-2
stacks:
us-east-1:
my-stack-name:
template: my-cf-stack.json
ap-southeast-2:
my-stack-name:
template: my-cf-stack.json```
from stack_master.
As per the README, regions can be referred to by their alias:
region_aliases:
production: us-east-1
staging: ap-southeast-2
...
stacks:
production:
myapp-vpc:
template: myapp_vpc.rb
I think the purpose of #115 and how I imagine it should work is you can have mutliple aliases using the same region:
region_aliases:
production: us-east-1
staging: us-east-1
...
stacks:
production:
myapp-vpc:
template: myapp_vpc.rb
staging:
myapp-vpc:
template: myapp_vpc.rb
These stacks would use the same template but the parameter folder lookup would be different. Production would be:
[ "parameters/myapp-vpc.yml", "parameters/us-east-1/myapp-vpc.yml", "parameters/production/myapp-vpc.yml" ]
while staging would be:
[ "parameters/myapp-vpc.yml", "parameters/us-east-1/myapp-vpc.yml", "parameters/staging/myapp-vpc.yml" ]
from stack_master.
@patrobinson , right thats how I understand it as well. IMO the only reason to do that would be because of the parameters. My approach mentioned above would accept a hash for the alias with a parameter which does control the behavior of how parameterfiles would be considered. The method with the prefix appears to be quite complex to achieve this. My approach would be more straightforward (IMO)
from stack_master.
My 2cs here.
It looks like the project started off with the assumption that different stacks would always exist in different regions. Hence the regions being effectively the primary key.
I think many of us have a different use case. We think in terms of environments. For example I have a an Alexa based project. The bits and pieces I need only exist in us-east-1. So I need production, staging etc all to exist in us-east-1.
I think we need to turn the current model on it's head and think about being able to have multiple environments in a region as a core use case. What I'm suggesting is a pretty major change and most likely requires a 1.0 release. My thinking is a config format something along the lines of
template_compilers:
rb: cfndsl
stack_defaults:
tags:
application: echohub
stacks:
web:
s3:
template: s3.rb
cloudfront:
template: cloudfront.rb
cicd:
ec2:
template: ec2.rb
environment_defaults:
region: us-east-1
environments:
prod:
stacks:
- web
staging:
stacks:
- web
test:
region: eu-west-1
stacks:
- web
- cicd
This would create the following stacks in us-east-1
- prod-web-s3
- prod-web-cloudfront
- stage-web-s3
- stage-web-cloudfront
In eu-west-1 it would create
- test-web-s3
- test-web-cloudfront
- test-cicd-ec2
from stack_master.
I'll start putting together a proof of concept, since I'll find something like this useful now.
from stack_master.
I've created the PoC and ended up with a config that looks like
template_compilers:
rb: cfndsl
region_defaults:
us-east-1:
secret_file: staging.yml.gpg
stack_defaults:
tags:
application: echohub
environment_defaults:
region: us-east-1
stacks:
s3:
template: s3.rb
pipeline:
template: pipeline.rb
cloudfront:
template: cloudfront.rb
acm:
template: acm.rb
route53:
template: route53.rb
environments:
production:
staging:
testing:
region: us-west-1
stacks:
cicd:
template: cicd.rb
from stack_master.
@patrobinson Doesn't this end up with a lot of duplication?
For example for my current project my production, staging and uat environments are all identical. With what I'm suggesting you can have a single stack_master.yml which looks like
stack_defaults:
tags:
application: echohub
environment_defaults:
region: us-east-1
stacks:
s3:
template: s3.rb
# Hundreds of lines here if needed
environments:
production:
staging:
testing:
I do understand that this is a pretty fundamental and breaking change that I'm suggesting, however I think it provides an improved workflow with less duplication.
From the parent directory I can do
stack_master apply production
stack_master apply staging
# etc
Anything I'm missing?
from stack_master.
@johnf That POC concept makes a lot of sense to me overall. How difficult would it be to add something to the environment to use different aws cli profiles? While there are a variety of mechanisms for credential management, supporting cli profiles would enable multiple account usage in a fairly transparent manner.
Multi-region deployments would be easy enough to setup with this design as additional environments as well.
from stack_master.
We might need to come up with some sort of extra syntax if we want the stack_ouput resolver to support this too.
Currently it is
vpc_id:
stack_output: [region:]stack-name/(OutputName|output_name)
Perhaps we should be more explicit?
vpc_id:
stack_output:
region: us-east-1
profile: account_b
stack_name: foo
output_name: bar
from stack_master.
I wouldn't try and add too much to your PR, getting stack outputs to work across accounts isn't really relevant
from stack_master.
I'd love some guidance here from the maintainers on their thoughts on this POC.
The main change I'm proposing is a breaking one, so if there is interest then we could start thinking about a 1.x roadmap create a branch etc and start adding these types of things in there.
I'm happy to help out and start breaking this up into appropriate PRs etc but there isn't much point unless the maintainers and other community members are on board.
from stack_master.
I wouldn't try and add too much to your PR, getting stack outputs to work across accounts isn't really relevant
Agree. Just dumping stuff in here for now as a POC, needs to be broken up if it moves forwards
from stack_master.
Any more interest in this?
from stack_master.
I think what johnf proposes makes a lot of sense. I also have use cases to deploy the same set/list of stacks each maintained as a separate CF template to a number of different accounts. These VPCs in these different account might or might not be in the same region. We are using 3 regions so far. So I like the approach of defining 'environments' which are a set/list of stacks if I understood this correctly
Right now I have to list the exact same set of stacks in each possible region just because some accounts use one region and others another.
region_aliases:
us: us-east-1
eu: eu-west-1
gov: us-gov-west-1
stacks:
us:
## new managed policy for Service Catalog users: cldauto, cldmgmt & cldarch depend on it
cataloguser-policy:
template: cataloguser-policy.cf.yml
## SVC role for System Manager Service. cldauto depend on it
cldauto-ssm-svc:
template: cldauto-ssm-svc.cf.yml
cldauto-fed:
template: cldauto-federated-role.cf.yml
cldmgmt-fed:
template: cldmgmt-federated-role.cf.yml
cldarch-fed:
template: cldarch-federated-role.cf.yml
cldwebops-fed:
template: cldwebops-federated-role.cf.yml
eu:
cataloguser-policy:
template: cataloguser-policy.cf.yml
cldauto-ssm-svc:
template: cldauto-ssm-svc.cf.yml
cldauto-fed:
template: cldauto-federated-role.cf.yml
cldmgmt-fed:
template: cldmgmt-federated-role.cf.yml
cldarch-fed:
template: cldarch-federated-role.cf.yml
cldwebops-fed:
template: cldwebops-federated-role.cf.yml
gov:
cataloguser-policy:
template: cataloguser-policy.cf.yml
cldauto-ssm-svc:
template: cldauto-ssm-svc.cf.yml
cldauto-fed:
template: cldauto-federated-role.cf.yml
cldmgmt-fed:
template: cldmgmt-federated-role.cf.yml
cldarch-fed:
template: cldarch-federated-role.cf.yml
cldwebops-fed:
template: cldwebops-federated-role.cf.yml
In this particular case I do not need to be passing different parameters to the stacks, but if I did they would have to be separated by 'account', not by region.
from stack_master.
Related Issues (20)
- Parameters are named with underscores, but stacks are named with dashes HOT 1
- Dependabot can't resolve your Ruby dependency files
- Dependabot can't resolve your Ruby dependency files
- Dependabot can't resolve your Ruby dependency files
- Dependabot can't resolve your Ruby dependency files
- Proposal: erb templating for parameter files HOT 2
- stack_master apply with no changes exits with non-zero HOT 2
- stack_master validate exits with non-zero on valid stack HOT 2
- Empty change list when only stack tags are changing HOT 1
- Add IAM capability 'CAPABILITY_AUTO_EXPAND' to apply.rb HOT 5
- General Inquiry on Directory Structure HOT 6
- Proposal: Add template_dir option to Stacks config HOT 1
- colorize gem is GPL-2 HOT 7
- --enable-termination-protection inquiry HOT 1
- Underscore in region alias causes apply to timeout (slowly)
- Logging-style output doesn't go to stderr
- Unclear error when missing permissions to validate allowed account
- Stack_master not getting installed on windows with Error as Failed to build gem native extension HOT 2
- Status command fails if parameters can't be resolved
- Current OnePassword integration does not support one password `op` cli v2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from stack_master.