data-for-change / dfc-iac

Data For Change infrastructure as code

Data For Change Infrastructure As Code

Prerequisites

• Python 3
• Vault CLI

Install Python Dependencies:

python3 -m pip install -r requirements.txt

Create a directory for secret files:

sudo mkdir -p /etc/dfc
sudo chown -R $USER /etc/dfc

Apps

The apps are defined using Docker Compose with some additional configurations under apps/

Anyway app is continuously deployed on any change to apps/anyway, the other apps need to be deployed manually.

To manage the apps you need to get a Vault TOKEN, access vault at https://vault.dataforchange.org.il and login. Once logged-in, click on your name in the top right corner and select "Copy token". Set the Token in an env var:

export VAULT_TOKEN=<paste token here>

You might need to get a new token occasionally, if you get an error message about an invalid token, just repeat the above steps.

Now you can run the apps management script to manage the apps, following are some examples:

# Run Docker Compose commands
bin/apps.py compose APP_NAME COMMAND
# for example:
bin/apps.py compose anyway ps
bin/apps.py compose anyway logs -f anyway-main

# SSH to the server
bin/apps.py ssh

# Run a command on the server
bin/apps.py ssh docker ps

# Deploy an app
bin/apps.py deploy_app APP_NAME

Terraform

Prerequisites:

• Terraform CLI
• AWS CLI

Running locally

Initialize (should only be done once):

terraform init "-backend-config=$(bin/get_backend_config.py)"

Set secret envvars:

eval "$(bin/get_secret_envvars.py)"

Check the plan:

terraform plan

Apply:

terraform apply

dfc-iac's Issues

look for additional infrastrcuture to migrate from hasadna

Cannot log in to server via ssh or see logs

I followed README instructions but didn't manage to perform basic commands:

➜  dfc-iac git:(main) bin/apps.py ssh
Traceback (most recent call last):
  File "bin/apps.py", line 204, in <module>
    main(*sys.argv[1:])
  File "bin/apps.py", line 196, in main
    res = globals()[cmd](*args)
  File "bin/apps.py", line 46, in ssh
    hostname = ssh_init()
  File "bin/apps.py", line 30, in ssh_init
    data = get_vault_kv_path('projects/iac/docker-server')
  File "bin/apps.py", line 24, in get_vault_kv_path
    'VAULT_TOKEN': VAULT_TOKEN,
  File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 411, in check_output
    **kwargs).stdout
  File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 488, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 800, in __init__
    restore_signals, start_new_session)
  File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'vault': 'vault'