I'm having issues running this in us-east-1 region. I get this error during the null_resource.install_docker_on_compose step

null_resource.install_docker_on_compose (remote-exec): Pulling hls    ... doneus: downloaded newer image fo...
null_resource.install_docker_on_compose (remote-exec):
null_resource.install_docker_on_compose (remote-exec): ERROR: for worker  Get https://[aws account number].dkr.ecr.us-east-1.amazonaws.com/v2/avalon/manifests/latest: no basic auth credentials

null_resource.install_docker_on_compose (remote-exec): ERROR: for avalon  Get https://[aws account number].dkr.ecr.us-east-1.amazonaws.com/v2/avalon/manifests/latest: no basic auth credentials
null_resource.install_docker_on_compose (remote-exec): ERROR: Get https://[aws account number].dkr.ecr.us-east-1.amazonaws.com/v2/avalon/manifests/latest: no basic auth credentials
null_resource.install_docker_on_compose (remote-exec): Get https://[aws account number].dkr.ecr.us-east-1.amazonaws.com/v2/avalon/manifests/latest: no basic auth credentials

Docker seems to be logging in fine with the aws ecr get-login command so I don't know what's going on here. Perhaps it's trying to push to this repo?

Wrong repository.

• Create certs using ACM
• Validate certs with Route 53
• Attach certs to ALB
• Update Avalon env

I am having issues building an environment while another environment already exists.

I am using s3 backend with separate workspaces, so environments should not be stepping on each other's toes.

I get this error when building the 2nd environment:

Error: error creating ECR repository: RepositoryAlreadyExistsException: The repository with name 'avalon' already exists in the registry with id '[aws account number]'

Perhaps something is hard coded to avalon when it should be avalon-[env]?

Description

For an actual deploy, we add it to the github repo; do a fork and put the file in the repo afterward. For someone spinning up terraform, what do we do?

Done Looks Like

• User provides controlled_vocabulary.yml, either in their fork or to the terraform script
• Submitted controlled_vocabulary.yml is deployed

Necessary for 7.2 otherwise files get stored in the Rails root which can get lost when container is destroyed or recreated.

Done looks like

Create

• SNS topic
• SMS subscription

Cloudwatch alarms for

• Health check
• CPU
• Memory
• Disk space
• Connections?

Only scaling Avalon/Worker/Streaming for now. Test scaling, task placement, etc.

When attempting to update a user via the "Manage Users" page (e.g. https://SITENAME/persona/users/1/edit ), clicking the "Update" button results in a 500 error ("We're sorry, but something went wrong").

Logs report a call to a missing update_attributes:

I, [2023-07-17T23:19:53.405056 #1]  INFO -- : [8a9c7b6b-2cfa-43b5-9184-71a6062c946b] Started PUT "/persona/users/1" for 35.8.223.97 at 2023-07-17 23:19:53 +0000
I, [2023-07-17T23:19:53.409020 #1]  INFO -- : [8a9c7b6b-2cfa-43b5-9184-71a6062c946b] Processing by Samvera::Persona::UsersController#update as HTML
I, [2023-07-17T23:19:53.409117 #1]  INFO -- : [8a9c7b6b-2cfa-43b5-9184-71a6062c946b]   Parameters: {"authenticity_token"=>"[FILTERED]", "user"=>{"email"=>"[email protected]", "username"=>"[email protected]", "password"=>"[FILTERED]", "password_confirmation"=>"[FILTERED]"}, "commit"=>"Update", "id"=>"1"}
I, [2023-07-17T23:19:53.437591 #1]  INFO -- : [8a9c7b6b-2cfa-43b5-9184-71a6062c946b] Completed 500 Internal Server Error in 28ms (ActiveRecord: 4.1ms | LDP: 0.0ms | Solr: 5.3ms | Allocations: 3861)
F, [2023-07-17T23:19:53.439724 #1] FATAL -- : [8a9c7b6b-2cfa-43b5-9184-71a6062c946b]   
[8a9c7b6b-2cfa-43b5-9184-71a6062c946b] NoMethodError (undefined method `update_attributes' for #<User id: 1, username: "[email protected]", email: "[email protected]", created_at: "2023-07-17 23:09:15.310030000 +0000", updated_at: "2023-07-17 23:13:00.793982000 +0000", provider: nil, uid: "[email protected]", guest: nil, deleted_at: nil>):
[8a9c7b6b-2cfa-43b5-9184-71a6062c946b]   
[8a9c7b6b-2cfa-43b5-9184-71a6062c946b] app/controllers/samvera/persona/users_controller.rb:132:in `block in update'
[8a9c7b6b-2cfa-43b5-9184-71a6062c946b] app/controllers/samvera/persona/users_controller.rb:131:in `update'
[8a9c7b6b-2cfa-43b5-9184-71a6062c946b] config/initializers/ldp_solr_duration_logger.rb:25:in `process_action'

Occurred on Avalon v7.6.0 installed via an avalon-terraform deployment.

Resetting password via the "Forgot your password" email link does work as expected.

Done looks like

• CodeBuild to build and push Docker image on Github trigger
• CodeDeploy to update demobox
• Make Github and Dockerhub repos configurable (optional)

Hello,

I am trying to run this in us-west-2 but the hard coded ami (ami-08b255f35f032a5ea) is us-east-1. I've tried using the ami_copy resource in terraform but it says I don't have access permissions on the ami

Errors for avalon and worker when trying to pull - no basic auth credentials

My experience with both docker-compose and AWS is limited and I'm not sure if I'm missing something I should know, or if this is an issue with the code. I have found a work around and am posting in the interest of helping others that may encounter this issue.

I am able to pull after logging in with the following when providing the specifics for my account # and the region:

aws ecr get-login-password --region (region) | docker login --username AWS --password-stdin (account #).dkr.ecr.(region).amazonaws.com

Description

Currently nginx uses IP-based policy to access S3. Would be better to use role-based policy with signed S3 request. s3-helper (crunchyroll.com) can do this transparently as an nginx upstream endpoint. Only caveat is it requires 127.0.0.0 origin so if we want it on a separate container, it needs forking.

https://medium.com/ellation-tech/just-in-time-vod-packaging-in-an-aws-environment-with-role-based-credentials-4e33202a4450

Done Looks Like

• Try out the role-based policy with signed S3 request per description above and create a new issue to implement in Turnkey if it works out

• Create S3 bucket
• Create cron job that backs up, zips and uploads to S3
• Test

Description

Removing custom AMI for flexibility in AWS environments. Also make it configurable.

Done Looks Like

• Remove custom AMI from terraform script
• Add customization

• Update compose's security group rules
• Add bastion ec2 instance

Current CodeBuild is for EC2, now that we're on ECS I'd like to make use of other features in CodeDeploy such as Blue-Green deployment.

Done looks like

• Fix CodeBuild script to use CodeDeploy
• Create CodeDeploy stages for Blue-Green deployment

Would be nice if horizontally scalable.

Postgres 10 will be end of life in November so this should be upgraded to 14 (which is used in the docker-compose.yml in avalon main). I wonder if we could also drop the custom parameter group because the only parameter declared here is already the default value.

Done Criteria

• Make a pull request to update Postgres
• Merge or try out running the script to ensure functionality.

Description

AWS presets rake task needs to be run when we first bring up the EC2 instance.

Done Looks Like

• Presets are created when terraform stack is stood up for the first time

ECS is not removing extra EC2 hosts after the load goes down, even though it is managed by the CapacityProvider.

Provisioning takes 3-4 minutes every time. We should try the latest build image (currently aws/codebuild/amazonlinux2-x86_64-standard:3.0) and see if it can go faster.

Also figure out better caching in the build step.

• ECR: image does not exist in the beginning (needs to run CodeBuild) #20
• Add configurable CIDR range for SSH
• Create presets when run scripts (2 points)
• Fix preset template (currently returning 2 extra nils)
• Create dropbox folder if not exist
• Add Manage Jobs to Admin view
• Add timeliner setting to Terraform script
• Missing controlled_vocabulary.yml (1 point)

/etc/docker/daemon.json

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m",
    "max-file": "3" 
  }
}

We're currently using a nginx container to cache assets but CloudFront is probably the better way to do it, less maintenance and more features.

Done looks like:

• Create CloudFront endpoint with ALB origin
• Disable nginx cache
• Rerun performance tests and compare