GithubHelp home page GithubHelp logo

cloudposse / terraform-aws-efs-backup Goto Github PK

View Code? Open in Web Editor NEW
43.0 17.0 33.0 1.08 MB

Terraform module designed to easily backup EFS filesystems to S3 using DataPipeline

Home Page: https://cloudposse.com/accelerate

License: Apache License 2.0

HCL 96.41% Makefile 3.59%
terraform terraform-modules datapipeline aws s3 efs nfs backup snapshot lambda

terraform-aws-efs-backup's People

Contributors

abferm avatar actions-user avatar aknysh avatar dylanbannon avatar josephchoe avatar max-lobur avatar osterman avatar plumdog avatar rfvermut avatar s2504s avatar solairerove avatar vadim-hleif avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

terraform-aws-efs-backup's Issues

parameter value for parameter name myKeyPair does not exist

Hi,

I have got following the errors applying (terraform apply):

module.efs_backup.aws_cloudformation_stack.datapipeline: 1 error(s) occurred: aws_cloudformation_stack.datapipeline: ROLLBACK_COMPLETE: ["Parameter validation failed: parameter value for parameter name myKeyPair does not exist. Rollback requested by user."]

I have used the following code:

main.tf

module "efs_backup" {
source = "git::https://github.com/cloudposse/terraform-aws-efs-backup.git?ref=master"

name = "${var.name}"
stage = "${var.stage}"
namespace = "${var.namespace}"
vpc_id = "${var.vpc_id}"
efs_mount_target_id = "${var.efs_mount_target_id}"
use_ip_address = "false"
noncurrent_version_expiration_days = "${var.noncurrent_version_expiration_days}"
ssh_key_pair = "${var.ssh_key_pair}"
datapipeline_config = "${var.datapipeline_config}"
modify_security_group = "true"
}

output "efs_backup_security_group" {
value = "${module.efs_backup.security_group_id}"

terraform.tfvars

namespace = "namespace"

stage = "stage"

name = "efs-backup"

region = "eu-central-1"

vpc_id = "vpc-0123456"

efs_mount_target_id = "fsmt-0123456"

#use_ip_address = "false"

#modify_security_group = "false"

noncurrent_version_expiration_days = "35"

ssh_key_pair = ""

#datapipeline_config = "${map("instance_type", "t2.micro", "email", "", "period", "24 hours", "timeout", "60 Minutes")}"

attributes = []

tags = {}

delimiter = "-"

variables.tf

variable "name" {
type = "string"
}

variable "namespace" {
type = "string"
}

variable "stage" {
type = "string"
}

variable "region" {
type = "string"
default = ""
description = "(Optional) AWS Region. If not specified, will be derived from 'aws_region' data source"
}

variable "vpc_id" {
type = "string"
}

variable "use_ip_address" {
default = "false"
}

variable "datapipeline_config" {
type = "map"

default = {
instance_type = "t2.micro"
email = "[email protected]"
period = "24 hours"
timeout = "60 Minutes"
}
}

variable "efs_mount_target_id" {
type = "string"
description = "EFS Mount Target ID (e.g. fsmt-279bfc62)"
}

variable "modify_security_group" {
default = "false"
}

variable "ssh_key_pair" {
type = "string"
}

variable "noncurrent_version_expiration_days" {
default = "35"
}

variable "delimiter" {
type = "string"
default = "-"
description = "Delimiter to be used between name, namespace, stage, etc."
}

variable "attributes" {
type = "list"
default = []
description = "Additional attributes (e.g. efs-backup)"
}

variable "tags" {
type = "map"
default = {}
description = "Additional tags (e.g. map('BusinessUnit,XYZ)"
}

I have set the SSH Public Key in this way:

ssh_key_pair = "ssh-rsa ABCDEF123456"
but got error applying:
module.efs_backup.aws_cloudformation_stack.datapipeline: 1 error(s) occurred: aws_cloudformation_stack.datapipeline: ROLLBACK_COMPLETE: ["Parameter validation failed: parameter value ssh-rsa ABCDDEF123456 for parameter name myKeyPair does not exist. Rollback requested by user."]
then I set the variable to "", but then came the following error:
module.efs_backup.aws_cloudformation_stack.datapipeline: 1 error(s) occurred: aws_cloudformation_stack.datapipeline: ROLLBACK_COMPLETE: ["Parameter validation failed: parameter value for parameter name myKeyPair does not exist. Rollback requested by user."]

But according to documentation, this variable is optional so actually the second solution should work..

Do you have an idea how to solve it?

Thanks in advance!

Add folder path to EFS targer

Hello,

I need to make backup of folders inside EFS because EFS size is too big and takes a lot of time to make full backup. So it would be great to have option to specify targed folder in EFS. Is it possible to add path inside EFS to backup from?

I tried to add:

  • in efs.tf
    data "aws_efs_mount_target_folder" "default" {
    mount_target_folder = "${var.efs_mount_target_folder}"
    }
  • in variables.tf
    variable "efs_mount_target_folder" {
    type = "string"
    default = "<target_folder>"
  • in cloudformation.tf
    myEFSHostFolder = "${var.efs_mount_target_folder}"
    -in templates/datapipeline.yaml
    add variable "$4" for target folder, place it after "$source":/, and add 4th scriptArgument with value "#{myEFSHostFolder}"

But got errors after "terraform plan or init" - Error: resource 'data.aws_efs_mount_target_folder.default' config: unknown variable referenced: 'efs_mount_target_folder'; define it with a 'variable' block
and
Error: resource 'aws_cloudformation_stack.datapipeline' config: unknown variable referenced: 'efs_mount_target_folder'; define it with a 'variable' block

What did I miss?
Or add please opportunity to specify folders for backup inside the EFS.

Best Regards.

Add Example Usage

what

  • Add example invocation

why

  • We need this so we can soon enable automated continuous integration testing of module

Add Example Usage

what

  • Add example invocation

why

  • We need this so we can soon enable automated continuous integration testing of module

The following resource(s) failed to create: [Topic]

Hi,

I have got following the errors applying (terraform apply):

module.efs_backup.aws_cloudformation_stack.sns: 1 error(s) occurred: aws_cloudformation_stack.sns: ROLLBACK_COMPLETE: ["The following resource(s) failed to create: [Topic]. . Rollback requested by user." "Invalid parameter: Endpoint (Service: AmazonSNS; Status Code: 400; Error Code: InvalidParameter; Request ID: 0f1e8da9-7469-5dfd-8e5e-de14ff3509fa)"]

and afterwards applying it once again pops up the following:

module.efs_backup.aws_cloudformation_stack.datapipeline: 1 error(s) occurred: module.efs_backup.aws_cloudformation_stack.datapipeline: At column 39, line 1: map "aws_cloudformation_stack.sns.outputs" does not have any elements so cannot determine type. in: ${aws_cloudformation_stack.sns.outputs["TopicArn"]}

I have used the following code:

main.tf

module "efs_backup" {
source = "git::https://github.com/cloudposse/terraform-aws-efs-backup.git?ref=master"

name = "${var.name}"
stage = "${var.stage}"
namespace = "${var.namespace}"
vpc_id = "${var.vpc_id}"
efs_mount_target_id = "${var.efs_mount_target_id}"
use_ip_address = "false"
noncurrent_version_expiration_days = "${var.noncurrent_version_expiration_days}"
ssh_key_pair = "${var.ssh_key_pair}"
datapipeline_config = "${var.datapipeline_config}"
modify_security_group = "true"
}

output "efs_backup_security_group" {
value = "${module.efs_backup.security_group_id}"

terraform.tfvars

namespace = "namespace"

stage = "stage"

name = "efs-backup"

region = "eu-central-1"

vpc_id = "vpc-0123456"

efs_mount_target_id = "fsmt-0123456"

#use_ip_address = "false"

#modify_security_group = "false"

noncurrent_version_expiration_days = "35"

ssh_key_pair = "ssh-rsa key"

#datapipeline_config = "${map("instance_type", "t2.micro", "email", "", "period", "24 hours", "timeout", "60 Minutes")}"

attributes = []

tags = {}

delimiter = "-"

variables.tf

variable "name" {
type = "string"
}

variable "namespace" {
type = "string"
}

variable "stage" {
type = "string"
}

variable "region" {
type = "string"
default = ""
description = "(Optional) AWS Region. If not specified, will be derived from 'aws_region' data source"
}

variable "vpc_id" {
type = "string"
}

variable "use_ip_address" {
default = "false"
}

variable "datapipeline_config" {
type = "map"

default = {
instance_type = "t2.micro"
email = ""
period = "24 hours"
timeout = "60 Minutes"
}
}

variable "efs_mount_target_id" {
type = "string"
description = "EFS Mount Target ID (e.g. fsmt-279bfc62)"
}

variable "modify_security_group" {
default = "false"
}

variable "ssh_key_pair" {
type = "string"
}

variable "noncurrent_version_expiration_days" {
default = "35"
}

variable "delimiter" {
type = "string"
default = "-"
description = "Delimiter to be used between name, namespace, stage, etc."
}

variable "attributes" {
type = "list"
default = []
description = "Additional attributes (e.g. efs-backup)"
}

variable "tags" {
type = "map"
default = {}
description = "Additional tags (e.g. map('BusinessUnit,XYZ)"
}

Do you have maybe an idea how to solve it?

Thanks in advance!

aws_cloudformation_stack.datapipeline resource does not produce an array

The "datapipeline_ids" output references the "aws_cloudformation_stack.datapipeline" as though it produces the array. This hasn't been the case since 8bc76ef.

Error: Error refreshing state: 1 error(s) occurred:

* module.efs_backup.output.datapipeline_ids: __builtin_StringToInt: strconv.ParseInt: parsing "DataPipelineId": invalid syntax in:

${aws_cloudformation_stack.datapipeline.*.outputs["DataPipelineId"]}

"current": [REMOVED] Defaults to current provider region if no other filtering is enabled

The module is failing and I am getting the following error:
Error: module.jenkins_efs_backup.data.aws_region.default: "current": [REMOVED] Defaults to current provider region if no other filtering is enabled

The cause of the issue is in here:

data "aws_region" "default" {
current = true
}

The solution is to remove the current = true since it is deprecated and throwing an error for Terraform v0.11.13.

Reference: https://www.terraform.io/docs/providers/aws/d/region.html

Resource 'aws_cloudformation_stack.sns' does not have attribute 'outputs.TopicArn' for variable 'aws_cloudformation_stack.sns.outputs.TopicArn'

Hi,

I'm trying to create EFS backups using this module but I keep running into the following error:

* module.efs_backup.output.sns_topic_arn: Resource 'aws_cloudformation_stack.sns' does not have attribute 'outputs.TopicArn' for variable 'aws_cloudformation_stack.sns.outputs.TopicArn'

The current configuration of the module is:

### Backup
module "efs_backup" {
  source = "git::https://github.com/cloudposse/terraform-aws-efs-backup.git?ref=0.8.0"

  name                = "${var.name}"
  stage               = "${var.stage}"
  namespace           = "${var.namespace}"
  vpc_id              = "${module.vpc.vpc_id}"
  ssh_key_pair        = "${aws_key_pair.backupadmin.key_name}"
  efs_mount_target_id = "${element(module.efs.mount_target_ids, 0)}"

  datapipeline_config = {
    instance_type = "t2.micro"
    email         = "[email protected]"
    period        = "24 hours"
    timeout       = "60 Minutes"
  }

  modify_security_group = "false"
}

What am I missing/doing wrong?

Add Example Usage

what

  • Add example invocation

why

  • We need this so we can soon enable automated continuous integration testing of module

Error: Missing resource instance key

Hello,

I want to do some tests with terraform-aws-efs-backup module in version 0.9.0. I have this code:
`module "efs_backup" {
source = "git::https://github.com/cloudposse/terraform-aws-efs-backup.git?ref=master"

name = "${var.name}"
stage = "${var.stage}"
namespace = "${var.namespace}"
vpc_id = "${var.vpc_id}"
efs_mount_target_id = "${var.efs_mount_target_id}"
use_ip_address = "false"
noncurrent_version_expiration_days = "${var.noncurrent_version_expiration_days}"
ssh_key_pair = "${var.ssh_key_pair}"
datapipeline_config = "${var.datapipeline_config}"
modify_security_group = "true"
}

output "efs_backup_security_group" {
value = "${module.efs_backup.security_group_id}"
}with these variables:variable "name" {
default = "jira-efs"
}

variable "stage" {
default = "stage"
}

variable "namespace" {
default = "namespace"
}

variable "vpc_id" {
default = "vpc-02c0d010183c3c1ce"
}

variable "efs_mount_target_id" {
default = "fs-2c057de4"
}

variable "noncurrent_version_expiration_days" {
default = "30"
}

variable "ssh_key_pair" {}

variable "datapipeline_config" {
type = "map"
default = {
instance_type = "t2.micro"
email = ""
period = "24 hours"
timeout = "60 Minutes"
}
}`

when I launched terraform plan I got this error multiple times with all the attributes:

`Error: Missing resource instance key

on .terraform/modules/efs_backup.backups_label/outputs.tf line 2, in output "id":
2: value = "${null_resource.default.triggers.id}"

Because null_resource.default has "count" set, its attributes must be accessed
on specific instances.

For example, to correlate with indices of a referring resource, use:
null_resource.default[count.index]`

I am pretty new with terraform, so probably I am missing something, but after readinng some posts I think it could be related with the new version of terrform (I am using version 0.12).

Thank you very much for your work!

Regards

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

These problems occurred while renovating this repository. View logs.

  • WARN: Base branch does not exist - skipping

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

Detected dependencies

terraform
cloudformation.tf
  • github.com/cloudposse/terraform-null-label tags/0.3.1
  • github.com/cloudposse/terraform-null-label tags/0.3.1
iam.tf
  • github.com/cloudposse/terraform-null-label tags/0.3.1
  • github.com/cloudposse/terraform-null-label tags/0.3.1
main.tf
  • github.com/cloudposse/terraform-null-label tags/0.3.1
s3.tf
  • github.com/cloudposse/terraform-null-label tags/0.3.1
  • github.com/cloudposse/terraform-null-label tags/0.3.1

  • Check this box to trigger a request for Renovate to run again on this repository

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.