GithubHelp home page GithubHelp logo

mybigman / homelab Goto Github PK

View Code? Open in Web Editor NEW

This project forked from brettinternet/homeops

0.0 1.0 0.0 855 KB

Resources for my homelab media server

Shell 12.82% HCL 4.99% Python 9.21% Makefile 4.62% Jinja 68.35%

homelab's Introduction

Homelab

Lint

Don't be fooled, having a home server is really just hundreds of hours of badblocks.

sudo badblocks -wsv -b 4096 /dev/sda output

I tried to fit as many buzzwords into this stack as I could: rootless Podman container orchestration with a ZFS array and SnapRAID JBOD, behind an OPNsense firewall and a Traefik ingress with OAuth, deployed with Ansible to multiple Arch Linux hosts, on a WireGuard network. ๐Ÿ…

This infrastructure as code is written for me because I'm forgetful. But perhaps it'll help you develop your own server architecture.

Setup

Run setup to create local configuration files and install requirements.yml from ansible-galaxy.

make setup

Then, edit inventory.yml with the target vars and secrets. See example.inventory.yml for what that looks like.

Deploy

If you're unfamiliar with Ansible, it's absolutely worth the effort to learn the mechanics and employ it in your own homelab.

Playbooks

See Working with playbooks and ansible-playbook

Bastion provision

Provision and setup a bastion server with a Digital Ocean Droplet. The setup creates a WireGuard server on the remote host and creates a client configuration on the home server. DNAT and SNAT traffic to and from the home server is routed through the bastion node with iptables.

Hosts setup and upgrade

  • Upgrade pacman and apt cache, packages and the apt distribution.
  • Deploy rootless containers in an orchestration behind Traefik's reverse proxy.
  • Setup SnapRAID for JBOD disk parity and configure cron to run a snapraid-runner script to sync parity and periodically check the data for errors.

Container composition

Rootless podman support for container images and deployment within an ansible role.

Hardware

book cover: Mommy, Why is There a Server is the House?

Homelab

Type Item
CPU Intel Core i7-7700 3.6 GHz Quad-Core Processor
CPU Cooler Noctua NH-L9i 33.84 CFM CPU Cooler
Motherboard Gigabyte GA-H270N-WIFI Mini ITX LGA1151 Motherboard
Memory Corsair Vengeance LPX 16 GB (2 x 8 GB) DDR4-3000 CL15 Memory
Case Fractal Design Node 804 MicroATX Mid Tower Case
Power Supply EVGA G2 550 W 80+ Gold Certified Fully Modular ATX Power Supply

Resources

Memory

Storage

Controller

I used a widely-known and inexpensive method to add additional SATA storage via a Host Bus Adapter (HBA). I purchased a Dell Perc H310 a long while back. Mine did come from overseas, but it turned out to be legit. This video shows how it can be flashed to an LSI 9211-8i IT (see also 1, 2).

Here are other recommended controllers.

2.5" drive stackers

These printable stackers are great for stacking SSDs in a homelab.

Home automation

Zigbee/Z-Wave

Software

Linux

Since I use Arch Linux as my daily driver, it's convenient for me to also use it to run my servers. Debian is a worthy alternative, however I find the stability of Arch to be sufficient for home use. See my linux notes and automation.

Podman

How do user IDs and rootless containers work?

SSH

Here's a nice convenience for setting up authorized_keys stored on Github or Gitlab:

curl https://github.com/<username>.keys -o authorized_keys

You could pipe the output to sed to only grab a specific line sed '4!d'.

Check disks

Here's a handy script to automatically test disks with badblocks and SMART: Spearfoot/disk-burnin-and-testing.

Testing disks takes a long time for larger drives, but it's worth it to be thorough before determining whether to make a return. This is a destructive test, so it's probably best to use /dev/disk/by-id to be certain you're targeting the correct drive.

Use tune2fs -l <partition> to identify the block size.

sudo badblocks -wsv -b 4096 /dev/sda > sda_badblocks.txt

Here's some additional advice from /r/DataHoarders.

JBOD

MergerFS is a union filesystem for pooling drives together. It's a great pair with SnapRAID. An alternative is SnapRAID-BTRFS.

mkdir /mnt/disk{1,2,3,4}
mkdir /mnt/parity1 # adjust this command based on your parity setup
mkdir /mnt/storage # this will be the main mergerfs mount point (a collection of your drives)

Mount drives to these folders, then add /etc/fstab entries by ID.

ls /dev/disk/by-id

You must also include an entry for the MergerFS union, such as:

/mnt/disk* /mnt/storage fuse.mergerfs allow_other,use_ino,cache.files=partial,dropcacheonclose=true,category.create=mfs,fsname=mergerfs,minfreespace=10G 0 0

See also perfectmediaserver: MergerFS

Remember, for data that's irreplaceable RAID is not a backup.

mkinitcpio

Be sure to add zfs and resume

HOOKS=(base udev autodetect modconf block filesystems keyboard zfs resume fsck)

Then, regenerate the image.

ZFS

Install zfs-dkms and zfs-utils, and be sure to have linux-headers installed for dkms to work.

OS Installation

Use Ventoy to bundle bootable ISO and IMG images on a single USB.

Media

For a media server, it's a good idea to understand digital video.

homelab's People

Contributors

brettinternet avatar

Watchers

James Cloos avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.