GitHubHelp

husamemadH/assessment

final documentation for infrastructure intern assessment

★ 0Forks 0ShellGitHub ↗Compare

README

System Infrastructure Assessment: Final Documentation

1. Environment Overview

  • Operating System: Ubuntu Linux
  • Hypervisor: QEMU/KVM managed via virt-manager
  • VM Specifications: 6 GB RAM, 4 CPU cores

2. Linux Distribution Research & Recommendation

Before beginning the deployment, I evaluated several Linux distributions including Arch Linux, Ubuntu, Debian, RHEL, and Rocky/AlmaLinux.

Final Recommendation: For this specific assessment environment (and general internal platform/developer use), Ubuntu is the optimal choice. It provides a massive ecosystem, predictable systemd behavior, and low-friction setup. While Arch Linux was my initial host OS, its rolling release model and AUR reliance make it too unstable for production infrastructure. Conversely, while RHEL is the gold standard for regulated enterprise data centers due to its 10-year lifecycle and compliance certifications, it introduces unnecessary subscription overhead for a simple internal platform.

3. QEMU/KVM Host Setup

Before provisioning the VM, QEMU and its dependencies were installed on the Arch Linux host:

Command:

sudo pacman -S qemu-full libvirt virt-manager dnsmasq

Issue Encountered:

error: failed to prepare transaction (could not satisfy dependencies)

:: installing nettle (4.0-1) breaks dependency 'nettle=3.10.2' required by lib32-nettle
:: installing nettle (4.0-1) breaks dependency 'libnettle.so=8-64' required by libcurl-gnutls
:: installing nettle (4.0-1) breaks dependency 'libnettle.so=8-64' required by wget

Resolution: Added the -Syu flag to force a full system sync and upgrade alongside the installation, resolving all dependency conflicts.

Post-install configuration:

sudo systemctl enable --now libvirtd
sudo virsh net-start default
sudo virsh net-autostart default
sudo usermod -aG libvirt $USER

The Ubuntu ISO was then downloaded from the official Ubuntu website and a VM was provisioned with 6 GB RAM and 4 CPU cores.

4. Task Breakdown & Execution

Task 1: User & Directory Permissions

Objective: Create an application user and secure a designated directory.

Point 1 — Create the application user

sudo useradd -s /usr/sbin/nologin appsvc
id appsvc

Output:

uid=1001(appsvc) gid=1001(appsvc) groups=1001(appsvc)

The -s /usr/sbin/nologin flag ensures the account cannot be used for interactive login, making it a service-only user.

Point 2 — Create and secure the directory

sudo mkdir -p /opt/platform-status
sudo chown appsvc:appsvc /opt/platform-status
sudo chmod 750 /opt/platform-status
sudo bash -c 'echo "This directory holds the files for the internal platform status page." > /opt/platform-status/README.txt'

Evidence — /opt listing:

ls -la /opt
total 12
drwxr-xr-x  3 root   root   4096 Jun  3 15:48 .
drwxr-xr-x 20 root   root   4096 Jun  3 15:41 ..
drwxr-x---  2 appsvc appsvc 4096 Jun  3 15:53 platform-status

Evidence — directory contents (sudo required since we are not logged in as appsvc):

sudo ls -la /opt/platform-status
total 12
drwxr-x--- 2 appsvc appsvc 4096 Jun  3 15:53 .
drwxr-xr-x 3 root   root   4096 Jun  3 15:48 ..
-rw-r--r-- 1 appsvc appsvc   34 Jun  3 15:53 README.txt

Note on Permissions (750): This octal value ensures the Owner (appsvc) has full Read/Write/Execute permissions (7), the Group (appsvc) has Read/Execute permissions (5), and all other unauthorized users have zero access (0).

Task 2: Networking and Service Exposure

Objective: Inspect network state, secure connectivity, and establish services.

Point 1 — Network IPs & Gateway

ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:51:15:02 brd ff:ff:ff:ff:ff:ff
    altname enx525400511502
    inet 169.254.239.207/16 brd 169.254.255.255 scope link noprefixroute enp1s0
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fe51:1502/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever

The address 169.254.x.x is a link-local fallback, meaning the interface failed to obtain an address from the DHCP server. To resolve this, the interface name was identified and a static IP was manually assigned within the virtual network's subnet.

nmcli connection show
NAME            UUID                                  TYPE      DEVICE
netplan-enp1s0  cac41fbe-bc18-3d87-bba7-af2af7f8ffab  ethernet  enp1s0
lo              c09a2266-c82d-4224-9c0e-f203fc46a720  loopback  lo

sudo nmcli connection modify "netplan-enp1s0" ipv4.addresses 192.168.122.50/24 ipv4.gateway 192.168.122.1 ipv4.dns "8.8.8.8,192.168.122.1" ipv4.method manual
sudo nmcli connection up "netplan-enp1s0"
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/3)

ip route
default via 192.168.122.1 dev enp1s0 proto static metric 20100
192.168.122.0/24 dev enp1s0 proto kernel scope link src 192.168.122.50 metric 100

The static IP 192.168.122.50 was successfully configured with gateway 192.168.122.1 (sourced from QEMU network settings).

Point 2 — Listening Ports & SSH

Running ss -tlpen before SSH installation revealed no port 22:

ss -tlpen
State   Recv-Q  Send-Q   Local Address:Port     Peer Address:Port  Process
LISTEN  0       4096         127.0.0.1:631           0.0.0.0:*      ino:16662 sk:8 cgroup:/system.slice/system-cups.slice/cups.service <->
LISTEN  0       4096     127.0.0.53%lo:53            0.0.0.0:*      uid:989 ino:4912 sk:9 cgroup:/system.slice/systemd-resolved.service <->
LISTEN  0       4096        127.0.0.54:53            0.0.0.0:*      uid:989 ino:4914 sk:a cgroup:/system.slice/systemd-resolved.service <->
LISTEN  0       4096             [::1]:631              [::]:*       ino:16661 sk:b cgroup:/system.slice/system-cups.slice/cups.service v6only:1 <->

SSH was not listening, so it was installed and enabled:

sudo apt update && sudo apt install openssh-server -y
sudo systemctl enable --now ssh

Running ss -tlpen again confirmed port 22 was now open:

State    Recv-Q   Send-Q     Local Address:Port     Peer Address:Port  Process
LISTEN   0        4096       127.0.0.53%lo:53            0.0.0.0:*      uid:989 ino:7030 sk:1 cgroup:/system.slice/systemd-resolved.service <->
LISTEN   0        4096          127.0.0.54:53            0.0.0.0:*      uid:989 ino:7032 sk:2 cgroup:/system.slice/systemd-resolved.service <->
LISTEN   0        4096           127.0.0.1:631           0.0.0.0:*      ino:17571 sk:3 cgroup:/system.slice/system-cups.slice/cups.service <->
LISTEN   0        4096             0.0.0.0:22            0.0.0.0:*      ino:27294 sk:5 cgroup:/system.slice/ssh.socket <->
LISTEN   0        4096               [::1]:631              [::]:*       ino:17570 sk:4 cgroup:/system.slice/system-cups.slice/cups.service v6only:1 <->
LISTEN   0        4096                [::]:22               [::]:*       ino:24553 sk:6 cgroup:/system.slice/ssh.socket v6only:1 <->

Point 3 — Outbound HTTPS Test

curl was installed to test outbound connectivity:

sudo apt install curl
curl -Iv https://google.com
* Host google.com:443 was resolved.
* IPv6: 2a00:1450:4006:809::200e
* IPv4: 172.217.20.110
*   Trying [2a00:1450:4006:809::200e]:443...
* connect to 2a00:1450:4006:809::200e port 443 from fec0::59dd:fbc6:7bb0:4c8e port 38416 failed: Network is unreachable
*   Trying 172.217.20.110:443...
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* SSL Trust Anchors:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
*   CApath: /etc/ssl/certs
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / X25519MLKEM768 / id-ecPublicKey
* ALPN: server accepted h2
* Server certificate:
*   subject: CN=*.google.com
*   start date: May  7 15:51:26 2026 GMT
*   expire date: Jul 30 15:51:25 2026 GMT
*   issuer: C=US; O=Google Trust Services; CN=WR2
*   Certificate level 0: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha384WithRSAEncryption
*   subjectAltName: "google.com" matches cert's "google.com"
* SSL certificate verified via OpenSSL.
* Established connection to google.com (172.217.20.110 port 443) from 10.0.2.15 port 60808
* using HTTP/2
...
< HTTP/2 301

The TLS handshake completed successfully via IPv4. IPv6 was unreachable within the VM network, but the connection fell back gracefully and succeeded.

Point 4 — Traceroute

traceroute was not pre-installed and was installed before use:

traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
 1  _gateway (10.0.2.2)       0.226 ms  0.211 ms  0.203 ms
 2  _gateway (192.168.100.1)  0.668 ms  0.657 ms  0.818 ms
 3  10.14.128.1               9.870 ms 10.014 ms 10.044 ms
 4  10.119.9.130             20.375 ms 20.364 ms 20.376 ms
 5  172.30.10.130             6.377 ms  6.236 ms  6.589 ms
 6  151.248.98.152            6.760 ms  5.861 ms  6.091 ms
 7  151.248.105.50           54.537 ms 54.856 ms 54.834 ms
 8  142.250.166.24          129.459 ms 129.356 ms 129.552 ms
 9  192.178.105.209          58.630 ms *          58.277 ms
10  142.251.78.85            63.039 ms 72.14.233.39 55.481 ms 142.251.78.83 262.874 ms
11  dns.google (8.8.8.8)     63.225 ms  63.804 ms  62.802 ms

The first hop outside the local virtual environment was hop 3 (10.14.128.1, ISP private routing). The first true hop to the public internet, outside the ISP's private network, occurred at hop 6 (151.248.98.152).

Concept: TCP vs UDP

  • TCP (Transmission Control Protocol): A connection-oriented, highly reliable protocol. It establishes a formal connection via a 3-way handshake and guarantees that all data arrives in order by requiring the receiver to acknowledge every packet. The tradeoff is added latency and overhead.
  • UDP (User Datagram Protocol): A connectionless, unreliable protocol. It prioritizes speed by blasting data to the receiver without checking if it arrived successfully. Ideal for live video streaming or gaming where speed matters more than perfect delivery.

Task 3: Bash Healthcheck Script

Objective: Create an executable script to monitor and log system parameters.

sudo nano /usr/local/bin/healthcheck.sh
sudo chmod +x /usr/local/bin/healthcheck.sh

Testing the Disk Threshold: To verify the >80% disk space warning functioned correctly, the script was temporarily edited to inject a mock variable (DISK_USAGE=99). The script successfully evaluated the condition and printed:

WARNING: Disk usage is above 80%!

Task 4: Systemd Automation

Objective: Automate the script to run every 5 minutes using systemd.

Configuration Files Created:

  1. /etc/systemd/system/healthcheck.service
[Unit]
Description=System Healthcheck Automation

[Service]
Type=oneshot
ExecStart=/usr/local/bin/healthcheck.sh

Type=oneshot tells systemd the script runs once and exits, rather than running as a persistent daemon.

  1. /etc/systemd/system/healthcheck.timer
[Unit]
Description=Run System Healthcheck Every 5 Minutes

[Timer]
# Run 1 minute after boot
OnBootSec=1min
# Run every 5 minutes thereafter
OnUnitActiveSec=5min

[Install]
WantedBy=timers.target

Activation:

sudo systemctl daemon-reload
sudo systemctl enable healthcheck.timer
sudo systemctl start healthcheck.timer

Verification — timer status:

systemctl list-timers --all | grep healthcheck
Fri 2026-06-05 16:22:37 +03  4min 38s  Fri 2026-06-05 16:17:37 +03  21s ago  healthcheck.timer  healthcheck.service

systemctl status healthcheck.timer
● healthcheck.timer - Run System Healthcheck Every 5 Minutes
     Loaded: loaded (/etc/systemd/system/healthcheck.timer; enabled; preset: enabled)
     Active: active (waiting) since Fri 2026-06-05 16:17:37 +03; 1min 32s ago
 Invocation: 1714c636f87c45859b6b71d180ca15f9
    Trigger: Fri 2026-06-05 16:22:37 +03; 3min 27s left
   Triggers: ● healthcheck.service

Jun 05 16:17:37 husam-Standard-PC-Q35-ICH9-2009 systemd[1]: Started healthcheck.timer - Run System Healthcheck Every 5 Minutes.

Journal Logs:

journalctl -u healthcheck.service -n 30 --no-pager
Jun 05 16:17:37 husam-Standard-PC-Q35-ICH9-2009 systemd[1]: Starting healthcheck.service - System Healthcheck Automation...
Jun 05 16:17:37 husam-Standard-PC-Q35-ICH9-2009 healthcheck.sh[7483]: ===== System Healthcheck =====
Jun 05 16:17:37 husam-Standard-PC-Q35-ICH9-2009 healthcheck.sh[7483]: Host: husam-Standard-PC-Q35-ICH9-2009
Jun 05 16:17:37 husam-Standard-PC-Q35-ICH9-2009 healthcheck.sh[7483]: Timestamp: 2026-06-05 16:17:37
Jun 05 16:17:37 husam-Standard-PC-Q35-ICH9-2009 healthcheck.sh[7483]: Uptime: up 18 minutes
Jun 05 16:17:37 husam-Standard-PC-Q35-ICH9-2009 healthcheck.sh[7483]: Load Average: 1.25 0.54 0.34
Jun 05 16:17:37 husam-Standard-PC-Q35-ICH9-2009 healthcheck.sh[7483]: CPU: 30.2% used
Jun 05 16:17:37 husam-Standard-PC-Q35-ICH9-2009 healthcheck.sh[7483]: Memory: total=5302MB used=1594MB free=638MB
Jun 05 16:17:37 husam-Standard-PC-Q35-ICH9-2009 healthcheck.sh[7483]: Disk: 57% used
Jun 05 16:17:37 husam-Standard-PC-Q35-ICH9-2009 healthcheck.sh[7483]: OK
Jun 05 16:17:37 husam-Standard-PC-Q35-ICH9-2009 healthcheck.sh[7483]: Top memory processes:
Jun 05 16:17:37 husam-Standard-PC-Q35-ICH9-2009 healthcheck.sh[7483]: PID COMMAND          %MEM
Jun 05 16:17:37 husam-Standard-PC-Q35-ICH9-2009 healthcheck.sh[7483]: 2027    gnome-shell     6.9
Jun 05 16:17:37 husam-Standard-PC-Q35-ICH9-2009 healthcheck.sh[7483]: 5372    ptyxis          4.5
Jun 05 16:17:37 husam-Standard-PC-Q35-ICH9-2009 healthcheck.sh[7483]: 6686    unattended-upgr 3.7
Jun 05 16:17:37 husam-Standard-PC-Q35-ICH9-2009 systemd[1]: healthcheck.service: Deactivated successfully.
Jun 05 16:17:37 husam-Standard-PC-Q35-ICH9-2009 systemd[1]: Finished healthcheck.service - System Healthcheck Automation.

Task 5: Dockerized Internal Status Page

Objective: Deploy an Nginx web server via Docker mapping to port 8080.

Installation & setup:

sudo apt install docker.io -y
sudo systemctl enable --now docker

Creating the HTML page:

mkdir -p ~/docker
echo "<h1>Internal Status Page</h1>
<p>Name: Husam</p>
<p>Hostname: $(hostname)</p>
<p>Date: $(date)</p>" > ~/docker/index.html

Running the container:

sudo docker run -d \
  --name status-page \
  -p 8080:80 \
  --restart always \
  -v ~/docker/index.html:/usr/share/nginx/html/index.html:ro \
  nginx:alpine

Evidence — container running:

sudo docker ps
CONTAINER ID   IMAGE          COMMAND                  CREATED          STATUS          PORTS                                     NAMES
641ee0ad7342   nginx:alpine   "/docker-entrypoint.…"   34 seconds ago   Up 32 seconds   0.0.0.0:8080->80/tcp, [::]:8080->80/tcp   status-page

Evidence — page served correctly:

curl http://localhost:8080
<h1>Internal Status Page</h1>
<p>Name: Husam</p>
<p>Hostname: husam-Standard-PC-Q35-ICH9-2009</p>
<p>Date: Fri Jun  5 04:28:29 PM +03 2026</p>

Docker Concepts:

  • Image vs. Container: An image is a read-only blueprint containing the OS and application dependencies (analogous to a class in OOP). A container is the live, running instance spawned from that blueprint (analogous to an object).
  • Bind Mount vs. Volume: A bind mount maps a specific file or folder from the host directly into the container — used above to inject the HTML file. A volume is an isolated storage space managed entirely by Docker, best suited for persistent data such as databases.
  • Port Conflicts: If port 8080 were already in use, the conflicting process would be identified with ss -tlpen | grep 8080, then either killed, stopped, or worked around by remapping the container to a different host port (e.g., -p 8081:80).

5. Retrospective & System Tuning

Mistakes and Troubleshooting:

  • Arch Linux Dependency Issue: While installing QEMU on the Arch host, a dependency conflict arose involving libnettle.so=8-64. This was resolved by using the pacman -Syu flag to force a full system sync and upgrade alongside the installation.

What I Had To Look Up: The correct package name for Docker on Ubuntu and how to set up QEMU/KVM from scratch.

Future Automations: If this were a real production environment, all manual terminal steps would be eliminated by provisioning the VM using Terraform with the QEMU/libvirt provider, turning the entire infrastructure into version-controlled, repeatable code.

6. Final Verification

The system is fully operational. A final review confirms:

  1. ss -tlpen shows ports 22 (SSH) and 8080 (Docker/Nginx) actively listening.
  2. systemctl list-timers shows healthcheck.timer ticking down and firing every 5 minutes.
  3. curl localhost:8080 returns the dynamically generated HTML status page.

Contributors

husamemadH

Issues