This project puts together a public network containing an auto scaled group of Squid Proxy servers configured to access the internet using NAT Gateways and an Internet Gateways. A private VPC is also created and peered with the public VPC with proxy traffic routed to the Squid Proxy Elastic Load Balancer. The provisioning of all the infrastructure and services is done using Terraform. All compute servers use T2 micros with Amazon Linux, note that the architecture shown below requires a region with a minimum of 3 availability zones.
The Squid Proxy has been configured for testing purposes. The following changes have been made to the configuration found in /etc/squid/squid.conf
The network can be configured to accept single IP addresses or CIDR ranges. In our case we grant access all local traffic in the public and private VPC
acl localnet src 10.1.0.0/16
acl localnet src 10.2.0.0/16
We can control access to domains and websites, in our example we control access to the AWS SQS service and the Amazon AWS website it self.
acl whitelist dstdomain sqs.us-east-1.amazonaws.com
acl whitelist dstdomain sqs.us-east-2.amazonaws.com
acl whitelist dstdomain sqs.us-west-1.amazonaws.com
acl whitelist dstdomain sqs.us-west-2.amazonaws.com
acl whitelist repo.us-east-1.amazonaws.com
acl whitelist repo.eu-west-1.amazonaws.com
acl whitelist repo.eu-west-2.amazonaws.com
acl whitelist packages.us-east-1.amazonaws.com
acl whitelist packages.eu-west-1.amazonaws.com
acl whitelist packages.eu-west-2.amazonaws.com
acl whitelist dstdomain www.amazonaws.com
You will need the following tools and accounts to make it happen
You will need an AWS account, if you don't have one then just Sign-Up You will need a development user with admin access and security access keys in order to use the AWS CLI
Install the AWS CLI and set-up a profile using your access keys
You will need to set-up your desired AWS region with a secure key pair. This project is using London (eu-west-2) as the region I suggest you keep this the same as there are AMI's referenced that are region sensitive.
EC2 Key Pairs has instructions on how to set-up your key pair.
Once you have set-up a key pair you will should have access to the PEM file that needs to be stored safely. Terraform assumes that the PEM is available in your local key chain, you can add it it to your key-chain by running the following command
ssh-add -K your-key.pem
Install terraform command line tool. Depends on your OS, Install Terraform has some descent instructions.
If you are using OSX I suggest you use Homebrew to install the terraform package.
I suggest that you run a plan to determine that access to your AWS account is as expected. Run the following command in the root of the terraform source src/main/terraform Note that it is recommended that you override the default values for aws_profile and the bastion_network_cidr
terraform plan -var "aws_key_name=your-key-name" -var "aws_region=us-east-1"
The output of this command will detail all the resources that will be created once applied. Note that it is recommended that you override the default values for aws_region (eu-west-1), aws_profile (default) and the bastion_network_cidr (0.0.0.0/0) e.g.
terraform plan -var "aws_key_name=you-key" -var "aws_profile=squid-lab" -var "bastion_network_cidr=199.250.200.0/24"
Once you are happy with the plan apply and decided on what variables to override you can apply your plan e.g.
terraform apply -var "aws_key_name=your-key"
The best way to test the application is to SSH on to the test host and attempt accessing the internet.
- Copy your EC2 key to the bastion host - scp -i <YOUR_KEY>.pem <YOUR_KEY>.pem ec2-user@<BASTION_HOST_IP>:.ssh/
- SSH to the bastion host - ssh ~/.ssh/<YOUR_KEY>.pem ec2-user@<BASTION_HOST_IP>
- SSH to the application host - ssh ~/.ssh/<YOUR_KEY>.pem ec2-user@<APPLICATION_HOST_IP>
- cURL a valid website; you should see output that looks like the following
>curl -I http://www.amazonaws.com
HTTP/1.1 302 Found
...
Via: 1.1 ip-10-1-3-240 (squid/3.5.20)
Connection: keep-alive
- cURL an invalid website; you should see output that looks like the following
>curl -I http://www.google.com
HTTP/1.1 403 Forbidden
Server: squid/3.5.20
...
Via: 1.1 ip-10-1-3-240 (squid/3.5.20)
Connection: keep-alive
The above test proves that access to the internet fromthe private VPC hosted application server can only be achieved through the white listed proxy. To prove this you could remove the export for the http_proxy settings and retry cURL test above; you will find that it timesout because there is no valid route to the internet gateway.
Clean up your environment by detroying, this will remove all traces:
terraform destroy -var 'aws_key_name=your-key.pem'
TODO - add FAQ for potential issues
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent