mrzzy / nimbus Goto Github PK
View Code? Open in Web Editor NEWSelf-hosted services in the Cloud.
License: MIT License
Self-hosted services in the Cloud.
License: MIT License
Linode provides cheaper infrastructure costs as compared to Google Cloud. Since this is a hobbyist side-project:
While moving to Linode from GCP would mean that we trade off:
Migrate Cloud Infrastructure from GCP to Linode Cloud:
Linode / Akamai is hiking their prices by 20% on 1st Apr.
Reverse course on #51.
Migrate Linode / Akamai deployments to GCP to integrate Nimbus deployments behind a single cloud:
WARP development VM on GCP currently has a development volume attached to it but it has to manually formatted & mounted.
/etc/fstab
.Manually setting up and tearing down WARP Box is non-reproducible & time consuming.
Just leaving the box running 24/7 will incur significant cloud provider billing.
Write a Terraform module to automate a reproducible deployment of the WARP Box on GCP.
Requirements
Run a instance with the warp-box
VM image built in mrzzy/warp#7
e2-standard-2
machine type.Enroll personal ssh key for SSH access into WARP box.
Harden networking with VPC Firewall Rules
Manually triggered CI Workflow to start WARP box VM.
Automated CI Workflow to destroy expense cloud resources (eg. VM instance) at 2230 daily.
Deploy a internet proxy for censorship circumvention:
Develop a monitoring solution to verify Internet Proxy circumventing ability.
sysctl
optimizations.Test deployments circumvention ability by:
Success: If blocking does not happen within 3 days.
Deploy shadowsocks-rust on Linode LKE following this guide
Deploy test Instance in Alibaba Cloud in China:
/metrics
endpoint in Prometheus format.Configure Prometheus to:
Deploy TiddlyWiki to host personal notetaking.
Deploy TiddyWiki on NodeJS:
Authentication flow:
Backup flow:
rclone
to copy tiddler files from sftp to Wasabi S3.wiki.mrzzy.co
DNS CNAME and point it at Ingress IPIn this CD step, we can see that kubectl
only acknowledges applying one Argo CRD:
appproject.argoproj.io/nimbus unchanged
Since the directory structure of k8s/argocd
changed in 306343b,
kubectl
has only applied the manifests in the top level directory only, not recursing to sub-directories.
Add the --recursive
flag to kubectl apply
in the CD when applying manifests to do so recursively.
Currently we use detect-secrets & pre-commit provide a pre commit secrets check:
detect-secrets falsely falsely flags SealedSecrets as secrets:
As a workaround, currently detect-secrets is configured by pre-commit to ignore all K8s manifests with -sealed.yaml
suffix:
nimbus/.pre-commit-config.yaml
Lines 18 to 20 in a5ff4a7
The workaround is brittle (relies on filename suffix). Additionally, ignoring secrets detection should be done mindfully require an explicit step.
Use secrets.baseline to manually configure secrets detection:
Currently, we deploy kube-prometheus-stack via a kustomization that basically boils down to a list of coded resource URLs.
Switch to jsonnet based deployment:
There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.
Location: renovate.json
Error type: The renovate configuration file contains some invalid settings
Message: Regex Managers must contain currentValueTemplate configuration or regex group named currentValue, Regex Managers must contain currentValueTemplate configuration or regex group named currentValue
Fix Accessing Webpages hosted on WARP Box with custom routes
Switch to host based routing instead of the, current, path based routing:
warp.mrzzy.co/spark
as spark.warp.mrzzy.co
instead.template.py
& nginx.conf.jinja2
to host based routingAt the time of writing, rtorent & flood are deployment together in the same container. Flood is responsible for managing the rtorrent instance.
This deployment mode has the following issues:
kubectl exec
the container and inspecting the log manually.Switch to deploying flood in separate containers.See example in Github Discussion.
rtorrent.rc
to work running standalone jesec/rtorrent
container.rtorrent.rc
config & K8s Service.rtorrent-flood
Deployment with pod with 2 containers, communicating via socket via shared volume (eg. cache).apiVersion
for the resources changes.apiVersion
and kind: Kustomization
not present in Kustomizations. Not pegging to a version for Kustomization resource might cause it to unexpectedly break in the future if the version of Kustomization that we depend on is dropped.apiVersion
and kind: Kustomization
to all under k8s/kustomize
.Post mrzzy/warp#17, WARP VM is able to expose a web accessible terminal via ttyd
.
However its currently locked down by VPC firewall due to security concerns (unencrypted HTTP)
Support the change of apporach on the WARP VM side.
Allow secure HTTPS web browser only access to WARP VM:
vm.warp.mrzzy.co
DNS route via GCP Cloud DNS with WARP VM's IP address.dns-01
ACME challenge type with GCP Cloud DNS.A self hosted web proxy will be useful to bypass overly restrictive network restrictions (both stackoverflow.com & github.com) are blocked.
We can't just deploy a Web Proxy on K8s and call it a day:
Piggyback on *.appspot.com
wildcard TLS certificate available for apps deployed on Google App Engine.
Deploy self-hosted Web Proxy on Google App Engine:
/var/log/app_engine/app.log
where it is access by Google App Engine.CNAME
DNS Alias proxy.mrzzy.co
to Proxy with Google App Engine provided URL: https://SERVICE_ID-dot-PROJECT_ID.REGION_ID.r.appspot.com
On a corporate network, HTTPS connections to made to WARP VM's Web Terminal is immediately met with a TCP reset.
Considered approaches:
Nginx Reverse Proxy Deploy Nginx on Google App Engine as WARP VM Proxy. This option requires significant engineering effort to implement.
*.appspot.com
certificates for services hosted on it. This certificate is trusted by the corporate network. Maintaining HTTPS will prevent DPI that the firewall relies on to block websockets needed by TTYD to work.Implementation
See mrzzy/warp#21
Add supporting infrastructure for WARP VM's HTTP terminal:
Refactor:
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
These updates have all been created already. Click a checkbox below to force a retry/rebase of any.
docker/naiveproxy/Dockerfile
caddy 2.8.4
.github/workflows/apply-terraform.yaml
actions/checkout v3
hashicorp/setup-terraform v2
ubuntu 20.04
.github/workflows/cleanup-terraform.yaml
actions/checkout v3
hashicorp/setup-terraform v2
actions/checkout v3
google-github-actions/auth v2
google-github-actions/setup-gcloud v2
ubuntu 20.04
ubuntu 20.04
.github/workflows/docker.yaml
actions/checkout v3
docker/login-action v2.2.0
docker/metadata-action v4.6.0
docker/build-push-action v6.7.0
ubuntu 22.04
.github/workflows/lint-secrets.yaml
actions/checkout v3
DariuszPorowski/github-action-gitleaks v2
ubuntu 20.04
.github/workflows/lint-terraform.yaml
actions/checkout v3
hashicorp/setup-terraform v2
actions/checkout v3
hashicorp/setup-terraform v2
ubuntu 20.04
ubuntu 20.04
requirements.txt
pre-commit ==3.8.0
.github/workflows/apply-terraform.yaml
hashicorp/terraform 1.9.5
.github/workflows/cleanup-terraform.yaml
hashicorp/terraform 1.9.5
.github/workflows/lint-terraform.yaml
hashicorp/terraform 1.9.5
hashicorp/terraform 1.9.5
terraform/aws.tf
terraform/azure.tf
terraform/b2.tf
terraform/cloudflare.tf
terraform/gcp.tf
github.com/mrzzy/warp 1bd4e73719565dfbcbfc7cccd1603e6bc9304edf
terraform/main.tf
acme 2.25.0
aws 4.67.0
azuread 2.53.1
azurerm 3.116.0
b2 0.8.12
cloudflare 4.40.0
google 4.85.0
hashicorp/terraform >=1.3.0
terraform/modules/cloudflare/dns/main.tf
cloudflare 4.40.0
terraform/modules/gcp/iam/main.tf
google >= 4.22.0
terraform/modules/gcp/vpc/main.tf
google >= 4.22.0
terraform/modules/tls_acme/main.tf
acme < 2.25.1
tls < 4.0.6
Deploy data engineering services required to work through the Shopee data engineering onboarding:
kubectl port-forward
is not unstable and unsuitable for accessing internal services.Currently, we embed sealed secrets in nimbus repository using the following workflow:
kubectl seal
, and commiting that.Secret
resources that deployments use.Pain points with this approach:
extraDeploy
s many helm charts have, but that requires needs alot more tooling.Must Haves:
Good to Haves:
Sealed secrets without generation from a single source of truth generation;
Github Secrets + Sealed Secrets + CD PIpeline:
kubeseal
(encrypt at rest).Hashicorp Vault + Vault Agent sidecar:
Out of Scope:
Deploy MariaDB with Bitnami Helm Chart
mariadb-root-password, mariadb-replication-password, mariadb-password
Improve the quality of video content consumed.
User flow of the streaming service:
media.mrzzy.co/files
to queue the Film / TV show for download.media.mrzzy.co
) to stream the Film / TV show.At steps 2. & 3. there should be some SSO login challenge to reject unauthorized users.
Kubernetes is introduced into the system design to mitigate the complexity of orchestrating multiple services.
flowchart LR
client([Jellyfin Client]) <--> ingress-nginx
subgraph K8s
direction LR
ingress-nginx <-->|auth.mrzzy.co| oauth2-proxy
ingress-nginx <-->|media.mrzzy.co| jellyfin[Jellyfin server]
ingress-nginx <-->|media.mrzzy.co/files| flood[FloodJS] <--> rtorrent
end
rtorrent -->|mount| bucket
bucket[(Storage Bucket)] -->|mount| jellyfin
Deploy a self-hosted Streaming Service:
Out of scope future work:
Different development tasks on requires a varying amount of system resources, no one size fits all machine type.
Line 80 in 15bf9bf
Allow WARP VM GCE machine type be configured on Deploy:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.