lancachenet / monolithic Goto Github PK
View Code? Open in Web Editor NEWA monolithic lancache service capable of caching all CDNs in a single instance
Home Page: https://hub.docker.com/r/lancachenet/monolithic
License: Other
monolithic's People
Contributors
Stargazers
Watchers
Forkers
stevesshop borzaka nyemenzo brainiarc7 jesgeerts midnighteyes zaiken64 cosmichorrordev mskafi mkimberley candunc synthbot dengalenubi ibigsnet riflejock120 lepidopterist russellwilson01 nxfifteen unspec crabbey decryptedchaos arnili poeticdeath didstopia evolvtyon thehaven rulerof cosito-bonito zorlin ipinternet itstkraz rptst ukkev mathewburnett mcm2007 lyon-esport kolinsmith shoryuag lancache-arm imliur nbh-network mezzovide koenvdheuvel spotlesscoder alexdrean snorfull mrflobow rekh127 b-reich sfinke0 docwilco rickyrsn prakhar64 njalooo joydenfew onethumb sat437 theraspb3rry askog96 mintopia htx-lan sweeny42 mohgitpr bergi9 andrewl3wis atesca09 bshiner jamestiotio handofiron neuroradiology tpill90monolithic's Issues
Trying to chain 2 caches, could use some advise
Describe the issue you are having
Hi, I was thinking about having like 10 caches , cache from a main cache as I have very little bandwidth, but a lot of devices and I wanted to speed this up as good as I can.
I made to test 2 vms ubuntu, 192.168.0.61 and 192.168.0.62. My lets say consumers use the lancache-dns on .61 which works fine. I set the upstream dns from .61 to 62. But when the cache was empty, it still downloaded the files directly to 61 without going through 62. Then I altered the network config of 61 to use 62 as only dns server. I tested it, and I was only able to resolve any hostnames when the 62 lancache-dns was running. But the cache is still probably using another dns server when downloading files for the cache.
Is there any way I could make it use the 62 dns server, so it would forward the download kinda to the second cache?
How are you running the container(s)?
192.168.0.61:
docker run --restart unless-stopped --name lancache -v /cache/data:/data/cache -v /cache/logs:/data/logs -p 192.168.0.61:80:80 lancachenet/monolithic:latest
docker run --restart unless-stopped --name lancache-dns -p 192.168.0.61:53:53/udp -e USE_GENERIC_CACHE=true -e LANCACHE_IP=192.168.0.61 -e CACHE_DOMAINS_REPO="https://github.com/FilixDark/cache-domains.git" -e UPSTREAM_DNS=192.168.0.62 lancachenet/lancache-dns:latest
docker run --restart unless-stopped --name lancache-sniproxy -p 192.168.0.61:443:443 lancachenet/sniproxy:latest
192.168.0.62
docker run --restart unless-stopped --name lancache -v /cache/data:/data/cache -v /cache/logs:/data/logs -p 192.168.0.62:80:80 lancachenet/monolithic:latest
docker run --restart unless-stopped --name lancache-dns -p 192.168.0.62:53:53/udp -e USE_GENERIC_CACHE=true -e LANCACHE_IP=192.168.0.62 -e CACHE_DOMAINS_REPO="https://github.com/FilixDark/cache-domains.git" lancachenet/lancache-dns:latest
docker run --restart unless-stopped --name lancache-sniproxy -p 192.168.0.62:443:443 lancachenet/sniproxy:latest
steamcache/monolithic for ARM based devices
The previous steamcache/steamcache repository works great with ARM-based devices (e.g. ODROID HC2 (has Gigabit LAN and a SATA-port, making it feasible for this service), but I am unable to get steamcache/monolithic to run on it because of the notorious exec error:
standard_init_linux.go:190: exec user process caused "exec format error"
This is solved by manually building the repo for steamcache/steamcache, but doesn't work for steamcache/monolithic. Is there any way I can get this to run on ARM?
Sincerely,
Kia.
Steam slow speed in the evening
Describe the issue you are having
First, sorry for my bad English.
I’m testing the cache monolith. My problem is, the Steam speed slows down dramatically in the evening. I tested it at 10:00 (UTC+1) and get my normal 5 MB/s but in the evening (18:00 +) it varies between 2 MB/s and 500 KB/s. Without the monolith it’s still 5 MB/s. Tested it with Origin too but no problem here.
Same problem yesterday. Same problem with generic or the original Steamcache.
My hardware isn’t maxed out at any point and after its cached I get up to 100 MB/s from the Server (CPU Limited).
Any idea?
Edit: Tested it with Battle.net and Uplay. Also, no problem.
How are you running the container(s)?
docker-compose.yml:
version: '3'
services:
cache:
container_name: cache
hostname: cache
image: steamcache/monolithic:latest
restart: unless-stopped
ports:
- 80:80
volumes:
- cache:/data/cache/
- logs:/data/logs/
- cachedomains:/data/cachedomains/
- www:/var/www/
networks:
nginx:
environment:
CACHE_MEM_SIZE: 4000m
CACHE_DISK_SIZE: 3500g
volumes:
cache:
external: true
logs:
cachedomains:
www:
networks:
nginx:
external: true
DNS Configuration
Running a Pi-Hole with the dnsmasq made files from uklans/cache-domains
Output of container(s)
[access.log](https://github.com/steamcache/monolithic/files/2866258/access.log)
[error.log](https://github.com/steamcache/monolithic/files/2866260/error.log)
Tell me if u need anything more and I will try to give it to you.
Managing Downloaded Files
Describe the issue you are having
Can you add a way to easily manage/view what's stored on the cache to make it easier to find and fix corrupted files as well as delete old/unused files.
How are you running the container(s)?
HOST_IP=`hostname -I | cut -d' ' -f1`
docker run --restart unless-stopped --name steamcache-dns --detach -p 53:53/udp -e USE_GENERIC_CACHE=true -e LANCACHE_IP=$HOST_IP steamcache/steamcache-dns:latest
docker run --restart unless-stopped --name lancache --detach -v /mnt/HDD1/cache/data:/data/cache -v /mnt/HDD1/cache/logs:/data/logs -p 80:80 -e CACHE_MEM_SIZE=12000m -e CACHE_DISK_SIZE=930g steamcache/monolithic:latest
docker run --restart unless-stopped --name sniproxy --detach -p 443:443 steamcache/sniproxy:latest
echo Please configure your dhcp server to serve dns as $HOST_IP
DNS Configuration
Google DNS
Xbox Live all pictures replaced by one image
Describe the issue you are having
Xbox Live all pictures replaced by one image
How are you running the container(s)?
docker run \
--restart unless-stopped \
--name cache-steam \
-v /cache/data:/data/cache \
-v /cache/logs:/data/logs \
-p 80:80 \
steamcache/monolithic:latest
DNS Configuration
Just put the server dns into my xbox and windows PC
I'm not sure if this is best posted here or in the domains list github.
Here's an image showing the problem
Static Or DHCP?
Should i used static ip address to be add in network interface or DHCP. if it static how many IP address should i add in it?
Rotate Downloads
Describe the issue you are having
Can you add an option to replace old data when the drive reaches a certain capacity e.g if the drive is nearly full start deleting/replacing the oldest files.
How are you running the container(s)?
HOST_IP=`hostname -I | cut -d' ' -f1`
docker run --restart unless-stopped --name steamcache-dns --detach -p 53:53/udp -e USE_GENERIC_CACHE=true -e LANCACHE_IP=$HOST_IP steamcache/steamcache-dns:latest
docker run --restart unless-stopped --name sniproxy --detach -p 443:443 steamcache/sniproxy:latest
echo Please configure your dhcp server to serve dns as $HOST_IP
DNS Configuration
Google DNS
Output of container(s)
<!-- Paste any output from the containers here - This will help us identify any issues faster -->
Confusing issue slow Blizzard vs fast Steam
Describe the issue you are having
Steam downloads of cached content max out the LAN connection. When downloading un-cached blizzard games like hearthstone i get maybe 1,5MB/s. When downloading hearthstone after its cached it maxes out at 30MB/s.
I dont understand how Steam is working 100% but I cant get Blizzard updates and games to go full speed.
I have tailed the error log and no errors were produced. The access log reports only HITS on the cache.
Im hoping its something stupid im doing.
How are you running the container(s)?
docker run --restart unless-stopped -d --name steamcache-multipleIPs -p 192.168.0.201:53:53/udp -v /etc/localtime:/etc/localtime -e STEAMCACHE_IP="192.168.0.202 192.168.0.203 192.168.0.204 192.168.0.205" -e BLIZZARDCACHE_IP="192.168.0.206 192.168.0.207 192.168.0.208 192.168.0.209 192.168.0.210" steamcache/steamcache-dns:latest
docker run --restart unless-stopped --name steamcache --detach -e UPSTREAM_DNS=192.168.0.9 -v /dload/steam/cache/data:/data/cache -v /dload/steam/cache/logs:/data/logs -p 192.168.0.202:80:80 -p 192.168.0.203:80:80 -p 192.168.0.204:80:80 -p 192.168.0.205:80:80 steamcache/monolithic:latest
docker run --restart unless-stopped --name blizzardcache --detach -e UPSTREAM_DNS=192.168.0.9 -v /etc/localtime:/etc/localtime -v /dload/blizzard/cache/data:/data/cache -v /dload/blizzard/cache/logs:/data/logs -p 192.168.0.206:80:80 -p 192.168.0.207:80:80 -p 192.168.0.208:80:80 -p 192.168.0.209:80:80 -p 192.168.0.210:80:80 steamcache/monolithic:latest
docker run --restart unless-stopped --name sniproxy --detach -p 443:443 steamcache/sniproxy:latest
DNS Configuration
Ethernet Connection Properties DNS -> Docker IP of .201
Output of container(s)
No Errors
Steam downloads normally, but items in cache disappear immediately.
Describe the issue you are having
First test of full stack container setup. Steam downloads games normally on the client, and files up to a few hundred KB appear briefly in cache, before disappearing and leaving an empty directory tree.
How are you running the container(s)?
docker run --name steamcache --restart=unless-stopped -d -v /mnt/four/steamcache:/data/cache -v /mnt/four/cachelog:/data/logs -p 80:80 steamcache/monolithic:latest
docker run --name steamcache-dns --restart=unless-stopped -d -p 53:53/udp -e UPSTREAM_DNS=1.1.1.1 -e LANCACHE_IP=192.168.1.133 -e USE_GENERIC_CACHE=true steamcache/steamcache-dns:latest
DNS Configuration
sudo docker run --restart=unless-stopped --name=sniproxy -d -p 443:443 steamcache/sniproxy:lates
Output of container(s)
Queries appear to all be handled normally (except steam has frequent/random problems logging in, eventually it works and fetches updates normally). However, steamcache is giving many chmod() permission errors in its log:
...
2019/05/13 06:08:38 [crit] 1263#1263: *200 chmod() "/data/cache/cache/8c/d3/f3a457e8084161061a5314c4af96d38c.0000000324" failed (1: Operation not permitted) while reading upstream, client: 192.168.1.125, server: , request: "GET /depot/426004/chunk/07bdc03d6bf9e2f57888b1274442ceacbb8d1f45?l=14&e=1558331878&sid=83492896&h=e8042f9033df88b5a63b07f2e2339e5676e16098 HTTP/1.1", upstream: "http://162.254.192.13:80/depot/426004/chunk/07bdc03d6bf9e2f57888b1274442ceacbb8d1f45?l=14&e=1558331878&sid=83492896&h=e8042f9033df88b5a63b07f2e2339e5676e16098", host: "cache21-iad1.steamcontent.com
...
Everything in the host's mapped cache directory (/mnt/four/steamcache) is owned by the primary user (UID=1000) with group root. Steamcache has created a huge empty directory tree under this, and if I spam a find to list all files in it, occasionally one will appear and immediately disappear again, leaving an empty tree again. For instance:
-rwxrwxrwx 1 paul root 20K May 13 02:14 /mnt/four/steamcache/cache/f9/24/154a71d3b7824dd0873875b667e524f9.0000000525*
Please let me know if additional container output is needed to diagnose this. Thanks!
CircleCI Build Children fails
Describe the issue you are having
Building children fails in circleci because monolithic has no build children step :S
New Xbox app for PC doesn't use cache
Describe the issue you are having
New Xbox app from Here does not use lancache
I've looked at the out of the container and theirs no activity when downloading from the new xbox app, So I assume that at the very least We need to work out what domains it's using to download.
How to install sniproxy within container?
Only getting 40Mbit-ish on initial caching
Describe the issue you are having
Admittedly I am still on the old steamcache-steam container, but before I go through the process of updating I wanted to see if I was missing something first.
Speeds when downloading a cached game are great. Easy 500Mbps when it gets up to speed. However, the initial speeds I'm getting for an uncached game on our 750Mbit line are a mere 45Mbit. Which isn't ideal.
I'd like to not kill steamcache, as it will be useful to have in the office (albeit it's a small team), so I'd like to try and resolve that one.
Is this just a case of write speeds not being sufficient, or should they not affect the speed of the download?
How are you running the container(s)?
version: '2'
services:
steamcache-steam:
container_name: steamcache-steam
hostname: steam.cache.domain.ext
image: steamcache/steamcache:latest
restart: unless-stopped
volumes:
- "/mnt/filestore/caches/steam/:/data/cache"
- "/mnt/filestore/logs/caches/steam:/data/logs"
ports:
- '80:80'
environment:
- TZ=Europe/London
networks:
default:
ipv4_address: 10.10.3.1
networks:
default:
external:
name: macvlan-nat
DNS Configuration
version: '2'
services:
steamcache-dns:
container_name: steamcache-dns
image: steamcache/steamcache-dns:latest
restart: unless-stopped
environment:
- TZ=Europe/London
- UPSTREAM_DNS=8.8.8.8
- STEAMCACHE_IP=10.10.3.1
networks:
default:
ipv4_address: 10.10.1.2
networks:
default:
external:
name: macvlan-nat
Unexpected status code 301 - officecdn.microsoft.com
Describe the issue you are having
Query for Microsoft Office return error in /cache/logs/error.log:
# /cache/logs/error.log:
2019/06/26` 17:32:08 [error] 1324#1324: *2337 unexpected status code 301 in slice response while reading response header from upstream, client: 192.168.11.213, server: , request: "GET /DB/492350f6-3a01-4f97-b9c0-c7c6ddf67d60/Office/Data/16.0.11727.20210/stream.x64.x-none.dat HTTP/1.1", subrequest: "/DB/492350f6-3a01-4f97-b9c0-c7c6ddf67d60/Office/Data/16.0.11727.20210/stream.x64.x-none.dat", upstream: "http://2.17.13.140:80/DB/492350f6-3a01-4f97-b9c0-c7c6ddf67d60/Office/Data/16.0.11727.20210/stream.x64.x-none.dat", host: "officecdn.microsoft.com"
This occurs when installing Office 2019 by online installer.
How are you running the container(s)?
export HOST_IP=`hostname -I | cut -d' ' -f1`
docker run --restart unless-stopped --name lancache-dns --detach -p 53:53/udp -e USE_GENERIC_CACHE=true -e LANCACHE_IP=$HOST_IP lancachenet/lancache-dns:latest
docker run --restart unless-stopped --name lancache --detach -v /cache/data:/data/cache -v /cache/logs:/data/logs -p 80:80 -e CACHE_DISK_SIZE=70g -e CACHE_MAX_AGE=180d lancachenet/monolithic:latest
docker run --restart unless-stopped --name sniproxy --detach -p 443:443 lancachenet/sniproxy:latest
DNS Configuration
# /etc/resolv.conf:
nameserver 8.8.8.8
nameserver 1.1.1.1
lancache is not caching most files
EDIT: So after so looking through the logs it does technically appear to cache "most" files but enough are left to to essentially remove 99% of any speed benefit.
I am having 3 problems the first one is the main one the other two are just questions:
Main problem:
lancache is not caching most steam files. I decided to try downloading a small game to test. The first time as expected the download ran a little slower (3MB/s instead of my usual 5MB/s - 8MB/s). The second time I went to download the speed went up to 5MB/s 20% "MISS". The third time it went to 15MB/s 10% "MISS", the fourth time it hit about 25MB/s 3% "MISS" and then finally on the fifth, sixth and seventh tries it capped out at 34.6MB/s <1% "MISS" which is great. The problem is if I have to download a game 5 times to get it to cache properly I haven't really saved any time over just leaving it in my original configuration.
I always redownloaded the same game without DLC or mods so that wasnt interfering with anything.
Questions:
So there was something mentioned about adding additional steam IPs? Are these actual steam server IPs or additional local IPs I can assign the server? If these are actual steam IPs could you explain where I can find these additional ones and how/where I put them. I'm sorry if it is a silly question but I can't for the life of me figure it out.
Here is a zip of all the log files cleared each run and attempted again with their DL rate in the title. I did this after I typed up the stuff above so DL rate may be slightly different.
https://www.mediafire.com/file/9nk7qz91j1iew91/steamcachelogs.zip/file
Also does this have any chance of killing my network connection? Literally to the point where unless I reapply my netplan I can no longer connect to the internet on my server because I had that happen once and I dont know if it was related or not.
Run script:
docker run --restart unless-stopped --name lancache-dns --detach -p 192.168.1.14:53:53/udp -e USE_GENERIC_CACHE=true -e LANCACHE_IP=192.168.1.14 lancachenet/lancache-dns:latest
docker run --restart unless-stopped --name lancache --detach -v /root/steam-cache:/data/cache -v /root/steam-cache:/data/logs -p 192.168.1.14:80:80 lancachenet/monolithic:latest
docker run --restart unless-stopped --name sniproxy --detach -p 192.168.1.14:443:443 lancachenet/sniproxy:latest
echo Please configure your router/dhcp server to serve dns as 192.168.1.14
*I removed the IP parsing bit and just directly put in my IP because it was not getting it properly. Also had to put my IP in for :53:53 because it was having trouble binding to the port.
Thanks.
Lancache Continuous Restart on 60 Second Cycle
Describe the issue you are having
Suspect these two issues may be related so posting together. Symptoms/Issues:
- Suggested install (from github instructions) breaks on Ubuntu Server. Ubuntu ships with systemd-resolved running by default. So lancache-dns will fail to start. I disabled systemd-resolve and this may be irrelevant to the main problem. But I'm not familiar with this so report it.
- Lancache goes into an immediate restart on launch, and continues restarting indefinitely. Unsure if it is related, but lancache is listening on port 443... That might be the problem, as sniproxy needs (and is using) that port. I assume this is coming from the Nginx base image, but could not figure out how to turn it off. Is it normal that lancache would be listening on this port too?
(See "Output of containers" for the container and ports list.)
How are you running the container(s)?
docker run --restart unless-stopped --name lancache-dns --detach \
-p 192.168.0.151:53:53/udp \
-p 172.17.0.1:53:53/udp \
-e USE_GENERIC_CACHE=true -e LANCACHE_IP="192.168.0.151 172.17.0.1" lancachenet/lancache-dns:latest
docker run \
--restart unless-stopped \
--name lancache \
--detach \
-v /cache/data:/data/cache \
-v /cache/logs:/data/logs \
-e UPSTREAM_DNS="1.1.1.1 1.0.0.1" \
-e CACHE_MEM_SIZE=4000m \
-e CACHE_DISK_SIZE=1t \
-p 192.168.0.151:80:80 \
-p 172.17.0.1:80:80 \
lancachenet/monolithic:latest
docker run --restart unless-stopped --name sniproxy --detach \
-p 192.168.0.151:443:443 \
-p 172.17.0.1:443:443 \
lancachenet/sniproxy:latest
DNS Configuration
Here is where I may have broken something... Because it wouldn't install due to port 53 being in use, I looked at three different sites explaining how to turn off systemd-resolved... The first part (stop and disable) worked. The second part (edit configs to apply manual DNS) was no-go... The configs referenced did not exist.
But, since I was installing a replacement DNS anyway, I continued. And lancache-dns appears to be working. I am posting this from a computer that got it's DNS from this instance of lancache-dns.
Output of container(s)
Insure what you mean, but here is a snapshot of the containers/ports in the single second befor lancache restarts:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f106a1f66b87 lancachenet/sniproxy:latest "/scripts/bootstrap.…" 34 seconds ago Up 32 seconds 172.17.0.1:443->443/tcp, 192.168.0.151:443->443/tcp sniproxy
e193af15121e lancachenet/monolithic:latest "/bin/bash -e /init/…" 44 seconds ago Up 1 second 172.17.0.1:80->80/tcp, 192.168.0.151:80->80/tcp, 443/tcp lancache
faa271c246dc lancachenet/lancache-dns:latest "/bin/bash -e /init/…" About a minute ago Up About a minute 172.17.0.1:53->53/udp, 192.168.0.151:53->53/udp lancache-dns
Steam missing cache
Describe the issue you are having
after moving from steamcache to monolithic i got no HIT. and prematurely closed error. i got this error. all of them. i triead all the games i have but no luck.
How are you running the container(s)?
i fork your mono and change env repo uklans/cache-domain to nyemenzo/cache-domain
DNS Configuration
steamcache dns. also change uklan/cache-domain to nyemenzo/cache-domain
Output of container(s)
*528 upstream prematurely closed connection while reading upstream, client: 192.168.8.100, server: , request: "GET /depot/731/chunk/36597b40d7290ee2ebe4cee5252d49759435258b?etime=1552520138&h=0893ee6b8582e7be1a502 HTTP/1.1", upstream: "http://67.24.11.252:80/depot/731/chunk/36597b40d7290ee2ebe4cee5252d49759435258b?etime=1552520138&h=0893ee6b8582e7be1a502", host: "level3.cdn.steampipe.steamcontent.com"
2019/03/07 00:36:33 [error] 220#220: *530 upstream prematurely closed connection while reading upstream, client: 192.168.8.100, server: , request: "GET /depot/731/chunk/c2749c4f76eb8b13a765e1cdf0a93eca1241d280?etime=1552520138&h=0893ee6b8582e7be1a502 HTTP/1.1", upstream: "http://67.24.13.252:80/depot/731/chunk/c2749c4f76eb8b13a765e1cdf0a93eca1241d280?etime=1552520138&h=0893ee6b8582e7be1a502", host: "level3.cdn.steampipe.steamcontent.com"
R+ games caching
Is there a possibility to cache rockstar games? Like when you got a RockstarGames key that you're able to cache that game too instead by using steam etc?
Documentation - Full Cache Behavior
The documentation isn't clear as to what occurs when the cache is full. Does it implement some eviction strategy (ie. LRU), or does it just not accept any new data until the max-age is reached on the existing data? I don't think anything needs to necessarily change with the container, but additional documentation on this matter may help others in deciding what cache values are appropriate for them.
Error Changing Upstream DNS
I'm receiving an error when trying to alter the upstream dns as instructed.
Running docker run --restart unless-stopped --name lancache-dns -p 53:53/udp -e USE_GENERIC_CACHE=true -e LANCACHE_IP=192.168.0.25 -e UPSTREAM_DNS="9.9.9.9 149.112.112.112" -e CACHE_M
AX_AGE=200d lancachenet/lancache-dns:latest
Returns this:
finished bootstrapping.
Executing hook /hooks/supervisord-pre.d/10_config_check.sh
checking Bind9 config
/etc/bind/named.conf.options:12: missing ';' before '149.112.112.112'
Problem with Bind9 configuration - Bailing
ERROR: hook /hooks/supervisord-pre.d/10_config_check.sh} returned a non-zero exit status '0'
Leaving out the upstream change allowed it to start correctly.
Hardware Requirements Documentation Suggestion
Describe the issue you are having
Readme correctly states that on typical commodity hardware and spinning disks you'll achieve up to 30MB/s. Exactly what I've experienced.
That's a little underwhelming, especially with typical home internet speeds in many cases exceeding that value. My suggestion is to list the hardware requirements necessary to achieve 1Gbit and 10Gbit throughput. Gigabit is perfect for a home lan-party, 10Gbit for commercial events.
If a cheap SSD can give me 100MB/s what the heck. If it takes whole new server I'll just throw in the towel. Some guidance would be greatly appreciated.
sniproxy is downloading new copies
Describe the issue you are having
Hello, I am having an issue when I am running the sniproxy with this current configuration that appears to be downloading new copies of game updates some place instead of in the lancache repo....not sure if this is an previous issue or not
How are you running the container(s)?
sudo docker run --restart unless-stopped --name steamcache-dns --detach -p 172.16.10.235:53:53/udp -e USE_GENERIC_CACHE=true -e LANCACHE_IP=172.16.10.235 steamcache/steamcache-dns:latest
sudo docker run --restart unless-stopped --name lancache --detach -v /mnt/lancache/data:/data/cache -v /mnt/lancache/logs:/data/logs -p 172.16.10.235:80:80 steamcache/monolithic:latest
sudo docker run --restart unless-stopped --name sniproxy --detach -p 172.16.10.235:443:443 steamcache/sniproxy:latest
Playstation 302 Header Errors
Describe the issue you are having
Receiving "unexpected status code 302 in slice response while reading response header from upstream" errors on playstation caching.
How are you running the container(s)?
docker run --restart unless-stopped --name lancache-dns --detach -p 53:53/udp -e USE_GENERIC_CACHE=true -e LANCACHE_IP=192.168.0.25 -e UPSTREAM_DNS="9.9.9.9;149.112.112.112" lancachenet/lancache-dns:latest
docker run --restart unless-stopped --name lancache --detach -v /home/user/cache/data:/data/cache -v /mnt/logs:/data/logs -p 80:80 -e UPSTREAM_DNS="9.9.9.9 149.112.112.112" lancachenet/monolithic:latest
docker run --restart unless-stopped --name lancache-sniproxy --detach -p 443:443 -e UPSTREAM_DNS="9.9.9.9 149.112.112.112" lancachenet/sniproxy:latest
DNS Configuration
Router is directing all systems to the DNS.
Output of container(s)
2019/10/02 08:37:59 [error] 1817#1817: *2484 unexpected status code 302 in slice response while reading response header from upstream, client: 192.168.0.226, server: , request: "GET /gs2/ppkgo/prod/CUSA07022_00/178/f_25375fa9ae011b3580d173f79416e588b81c8f20123de80188ea6a75c0612b1a/f/UP1477-CUSA07022_00-FORTNITETESTING1-A0238-V0100_0.pkg?downloadId=00000e38&du=0000000000000e3800e253e85c6e4184&product=0184&threadId=0&serverIpAddr=192.168.0.25&r=00000001&downloadType=om HTTP/1.1", subrequest: "/gs2/ppkgo/prod/CUSA07022_00/178/f_25375fa9ae011b3580d173f79416e588b81c8f20123de80188ea6a75c0612b1a/f/UP1477-CUSA07022_00-FORTNITETESTING1-A0238-V0100_0.pkg", upstream: "http://69.28.165.0:80/gs2/ppkgo/prod/CUSA07022_00/178/f_25375fa9ae011b3580d173f79416e588b81c8f20123de80188ea6a75c0612b1a/f/UP1477-CUSA07022_00-FORTNITETESTING1-A0238-V0100_0.pkg?downloadId=00000e38&du=0000000000000e3800e253e85c6e4184&product=0184&threadId=0&serverIpAddr=192.168.0.25&r=00000001&downloadType=om", host: "gs2.ww.prod.dl.playstation.net"
Allow for local development and testing of cache domains
Describe the issue you are having
When developing against a local instance of monolithic and bind mounting the cache-domains repository into the container, the map generation script always resets the repository state to origin/master.
How are you running the container(s)?
docker run --rm -v $PWD/code/uklans/cache-domains:/data/cachedomains lancachenet/monolithic
Ideally, there should be a single flag that disables the git fetch mechanism.
Can't clear /cache/data directory: Permission denied
Describe the issue you are having
I wanted to test the caching service, and as described in the readme, I tried to clear the /data/cache folder after a couple of days of running, but I couldn't over FTP.
CWD cache
550 Can't change directory to cache: Permission denied
RMD cache
550 Can't remove directory: Permission denied
DELE cache
550 Could not delete cache: Invalid argument
PWD
257 "/home/steamcache/cache/data" is your current location
How are you running the container(s)?
I have created a steamcache user, and added to the docker group.
HOST_IP=`hostname -I | cut -d' ' -f1`
docker run --restart unless-stopped --name steamcache-dns --detach -p 53:53/udp -e USE_GENERIC_CACHE=true -e LANCACHE_IP=$HOST_IP steamcache/steamcache-dns:latest
docker run --restart unless-stopped --name lancache --detach -v /home/steamcache/cache/data:/data/cache -v /home/steamcache/cache/logs:/data/logs -p 80:80 -e CACHE_MEM_SIZE=1000m -e CACHE_DISK_SIZE=500000m -e CACHE_MAX_AGE=365d -e LOGFILE_RETENTION=60 -e UPSTREAM_DNS="1.1.1.1 1.0.0.1" steamcache/monolithic:latest
docker run --restart unless-stopped --name sniproxy --detach -p 443:443 steamcache/sniproxy:latest
echo Please configure your dhcp server to serve dns as $HOST_IP
192.168.30.250 is my HOST_IP
steamcache@honved:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
89ffeaac571d steamcache/sniproxy:latest "sniproxy -f -c /etc…" 4 minutes ago Up 4 minutes 0.0.0.0:443->443/tcp sniproxy
6c331ecdfd23 steamcache/monolithic:latest "/bin/bash -e /init/…" 4 minutes ago Up 4 minutes 0.0.0.0:80->80/tcp, 443/tcp lancache
1930f4ec5cc1 steamcache/steamcache-dns:latest "bash /scripts/boots…" 4 minutes ago Up 4 minutes 0.0.0.0:53->53/udp steamcache-dns
Side question: what are the toughs of my custom environment variable settings? I don't understand some of the default values. 3650 days cache max age? 3650 days logfile retention? These are too much, aren't these?
DNS Configuration
Output of container(s)
The error.log is empty, and the access.log is huge as probably should be, this is an extract:
[wsus] 192.168.30.102 / - - - [13/Mar/2019:17:05:23 +0000] "GET /filestreamingservice/files/081ca1d9-1880-45d1-87f2-ec73c262409c?P1=1552497409&P2=402&P3=2&P4=QqBfyPl5aAsMeObyk4GU5%2bWayHjYpEl4i6IKme9BUSosV%2fQA6nFUDAkMnOl2B9jKMIqASBqqTKPPv88ZeHVL0w%3d%3d HTTP/1.1" 206 1014099 "-" "Microsoft-Delivery-Optimization/10.0" "HIT" "9.tlu.dl.delivery.mp.microsoft.com" "bytes=4194304-5208402"
[wsus] 192.168.30.109 / - - - [13/Mar/2019:17:05:28 +0000] "GET /filestreamingservice/files/b5c09e1f-7909-421a-8626-c8f759432571/pieceshash HTTP/1.1" 200 9460 "-" "Microsoft-Delivery-Optimization/10.0" "HIT" "dl.delivery.mp.microsoft.com" "-"
[wsus] 192.168.30.109 / - - - [13/Mar/2019:17:05:28 +0000] "GET /filestreamingservice/files/78ccaf0d-aa87-4d4a-bf7f-26080d4a223a/pieceshash HTTP/1.1" 200 9460 "-" "Microsoft-Delivery-Optimization/10.0" "HIT" "dl.delivery.mp.microsoft.com" "-"
[emdl.ws.microsoft.com] 192.168.30.109 / - - - [13/Mar/2019:17:05:29 +0000] "GET /emdl/c/doc/ph/prod6/msdownload/update/software/secu/2019/03/64/windows10.0-kb4489868-x64_ad95efe86b8219fef45fe886d3469dd07948a844.psf.json HTTP/1.1" 200 3907989 "-" "Microsoft-Delivery-Optimization/10.0" "HIT" "emdl.ws.microsoft.com" "-"
[wsus] 192.168.30.109 / - - - [13/Mar/2019:17:13:27 +0000] "GET /d/msdownload/update/software/defu/2019/03/am_delta_patch_1.289.1059.0_3cc1d82489f7648b65166b0824605db09484d842.exe HTTP/1.1" 206 2 "-" "Microsoft-Delivery-Optimization/10.0" "MISS" "au.download.windowsupdate.com" "bytes=0-1"
[wsus] 192.168.30.109 / - - - [13/Mar/2019:17:13:27 +0000] "GET /d/msdownload/update/software/defu/2019/03/am_delta_patch_1.289.1059.0_3cc1d82489f7648b65166b0824605db09484d842.exe HTTP/1.1" 206 334208 "-" "Microsoft-Delivery-Optimization/10.0" "HIT" "au.download.windowsupdate.com" "bytes=0-334207"
Linux distro
Debian 9 stretch 64-bit PC (amd64)
downloads slow and not caching at all
Describe the issue you are having
I set the DNS on the gaming PC to our test server 10.1.0.135. Downloading CS:GO, I see the network traffic go through the lancache. Initally, it was hitting 15-20 MB/s, but now after fiddling with a few things, it's down to 4-6 MB/s.
Even though it's downloading through the lancache, the disk usage doesn't increase at all. Even after downloading the game completey and deleting from the gaming PC, it seems to re-download through the cache, but again, not actually cache.
How are you running the container(s)?
This is the bash script that I made and run as sudo:
#!/bin/bash
echo stopping old container: steamcache-dns
docker stop steamcache-dns
echo stopping old container: lancache
docker stop lancache
echo stopping old container: sniproxy
docker stop sniproxy
echo deleting old container: steamcache-dns
docker rm steamcache-dns
echo deleting old container: lancache
docker rm lancache
echo deleting old container: sniproxy
docker rm sniproxy
HOST_IP=`hostname -I | cut -d' ' -f1`
echo set HOST_IP to: $HOST_IP
MEM_SIZE=4000m
DISK_SIZE=180g
echo starting: steamcache-dns
docker run --restart unless-stopped --name steamcache-dns --detach -p 53:53/udp -e USE_GENERIC_CACHE=true -e LA$
echo starting: lancache
echo LANCACHE MEMORY SIZE: $MEM_SIZE
echo LANCACHE DISK SIZE: $DISK_SIZE
docker run --restart unless-stopped --name lancache --detach -e CACHE_MEM_SIZE=$MEM_SIZE -e CACHE_DISK_SIZE=$DI$
echo starting: sniproxy
docker run --restart unless-stopped --name sniproxy --detach -p 443:443 steamcache/sniproxy:latest
echo Please configure your dhcp server to serve dns as $HOST_IPDNS Configuration
In the script above, this is how it's executed.
docker run --restart unless-stopped --name steamcache-dns --detach -p 53:53/udp -e USE_GENERIC_CACHE=true -e LA$
Output of container(s)
Output of the script I execute.
monolithic@monlithic:~$ sudo ./mono.sh
stopping old container: steamcache-dns
steamcache-dns
stopping old container: lancache
lancache
stopping old container: sniproxy
sniproxy
deleting old container: steamcache-dns
steamcache-dns
deleting old container: lancache
lancache
deleting old container: sniproxy
sniproxy
set HOST_IP to: 10.1.0.135
starting: steamcache-dns
268ba04fd222826ae12f8613e492b66b66042a6fdafb20bf17347eda52f5b74b
starting: lancache
LANCACHE MEMORY SIZE: 4000m
LANCACHE DISK SIZE: 180g
9ca10605d5b7fa2aba7b904eff444f03463f71b765eccb26d6b0b80a0837a960
starting: sniproxy
aedc8b894ec5ee648d04d122ff3069c9f8c34963b21bf5e96fd927635b78a9d8
Please configure your dhcp server to serve dns as 10.1.0.135
Also, I am curious. Would deleting the old containers get rid of the data on the lancache?
I was doing that because even though the commands to start the servers are docker restart, it yells at me that the containers already exist.
http/https splitting for monolithic
Describe the issue you are having
I'm currently investigating to switch to /monolithic from a /docker-compose like setup with multiple /generic instances
DNS Configuration
self-written powershell scripts to preload windows-server dns with the required dns redirects
As I'm in a windows-domain environment, I'm easily able to roll out trusted https certificates fof effective mitm caching of ssl content.
Currently I'm running the caching services on two different IPs, one for http only content, no ssl-certificates in nginx, and sni-proxy (just in case), and another IP that is running the nginx vhosts equipped with ssl certs, for origin and the likes.
I'm wondering how to adapt this setup to /monolithic. Are there any experiences regarding simply caching all the http/https traffic on the cdns listed by uklans/cache-domains, or should I explicitly whiltelist some of the services to ssl caching and route the rest through sni-proxy? I think I could achieve the latter by enrolling two IPs to the /monolithic container, listening in 80/443 on one, and listening on 80 with a running sni-proxy on the other IP.
Then binding these to different sets of host ips and using the according dns records for proper selection of ssl vs non-ssl caching for each cdn.
Are there any best-practices or known problems regarding such setups?
nginx: [emerg] chown("/data/cache/cache", 33) failed (13: Permission denied)
Describe the issue you are having
Some kind of permission issue I can't seem to figure out.
Once I start the container, it starts to restart in an infinite loop.
Running CentOS Linux release 7.6.1810 (Core)
Docker version 1.13.1, build b2f74b2/1.13.1
Directory permissions before:
total 0
0 drwxrwxrwx. 2 don tape 6 Jun 9 21:47 .
0 dr-xr-xr-x. 19 root root 251 Jun 9 20:19 ..Directory permissions after:
total 0
0 drwxrwxrwx. 3 don tape 19 Jun 9 21:49 .
0 dr-xr-xr-x. 19 root root 251 Jun 9 20:19 ..
0 drwxr-xr-x. 4 root root 30 Jun 9 21:49 cacheNo matter how many times I chown -R /shared to any user or group or permissions, the issue persists. Even tried a www-data group, as well as tape group, which has a GID of 33
How are you running the container(s)?
docker run --restart unless-stopped --name lancache -v /shared/cache/data:/data/cache -v /shared/cache/logs:/data/logs -p $LANCACHE_IP::80 lancachenet/monolithic:latest
DNS Configuration
Not Applicable, haven't gotten to this part yet.
Output of container(s)
docker run --restart unless-stopped --name lancache -v /shared/cache/data:/data/cache -v /shared/cache/logs:/data/logs -p $LANCACHE_IP::80 lancachenet/monolithic:latest
Executing hook /hooks/entrypoint-pre.d/10_setup.sh
Executing hook /hooks/entrypoint-pre.d/15_generate_maps.sh
Cloning into '.'...
HEAD is now at f6e7004 Merge pull request #90 from Crabbey/master
Reading cache apple from apple.txt
Reading cache arenanet from arenanet.txt
host: assetcdn.101.arenanetworks.com
new host: assetcdn.101.arenanetworks.com
host: assetcdn.102.arenanetworks.com
new host: assetcdn.102.arenanetworks.com
host: assetcdn.103.arenanetworks.com
new host: assetcdn.103.arenanetworks.com
Reading cache blizzard from blizzard.txt
host: dist.blizzard.com
new host: dist.blizzard.com
host: dist.blizzard.com.edgesuite.net
new host: dist.blizzard.com.edgesuite.net
host: llnw.blizzard.com
new host: llnw.blizzard.com
host: edgecast.blizzard.com
new host: edgecast.blizzard.com
host: blizzard.vo.llnwd.net
new host: blizzard.vo.llnwd.net
host: blzddist1-a.akamaihd.net
new host: blzddist1-a.akamaihd.net
host: blzddist2-a.akamaihd.net
new host: blzddist2-a.akamaihd.net
host: blzddist3-a.akamaihd.net
new host: blzddist3-a.akamaihd.net
host: blzddist4-a.akamaihd.net
new host: blzddist4-a.akamaihd.net
host: level3.blizzard.com
new host: level3.blizzard.com
host: nydus.battle.net
new host: nydus.battle.net
host: edge.blizzard.top.comcast.net
new host: edge.blizzard.top.comcast.net
host: cdn.blizzard.com
new host: cdn.blizzard.com
host: *.cdn.blizzard.com
new host: *.cdn.blizzard.com
Reading cache daybreak from daybreak.txt
host: # Daybreak games
new host: #Daybreakgames
host: #
new host: #
host: # Do NOT cache manifest.patch.daybreakgames.com
new host: #DoNOTcachemanifest.patch.daybreakgames.com
host: #
new host: #
host: #PS2
new host: #PS2
host: pls.patch.daybreakgames.com
new host: pls.patch.daybreakgames.com
Reading cache epicgames from epicgames.txt
host: epicgames-download1.akamaized.net
new host: epicgames-download1.akamaized.net
host: download.epicgames.com
new host: download.epicgames.com
host: download2.epicgames.com
new host: download2.epicgames.com
host: download3.epicgames.com
new host: download3.epicgames.com
host: download4.epicgames.com
new host: download4.epicgames.com
Reading cache frontier from frontier.txt
Reading cache hirez from hirez.txt
Reading cache nexusmods from nexusmods.txt
Reading cache nintendo from nintendo.txt
host: ccs.cdn.wup.shop.nintendo.com
new host: ccs.cdn.wup.shop.nintendo.com
host: pushmo.hac.lp1.eshop.nintendo.net
new host: pushmo.hac.lp1.eshop.nintendo.net
host: ecs-lp1.hac.shop.nintendo.net
new host: ecs-lp1.hac.shop.nintendo.net
host: receive-lp1.dg.srv.nintendo.net
new host: receive-lp1.dg.srv.nintendo.net
host: aqua.hac.lp1.d4c.nintendo.net
new host: aqua.hac.lp1.d4c.nintendo.net
host: atum.hac.lp1.d4c.nintendo.net
new host: atum.hac.lp1.d4c.nintendo.net
host: bugyo.hac.lp1.eshop.nintendo.net
new host: bugyo.hac.lp1.eshop.nintendo.net
host: tagaya.hac.lp1.eshop.nintendo.net
new host: tagaya.hac.lp1.eshop.nintendo.net
Reading cache origin from origin.txt
host: # WARNING: Origin has been seen downloading https client downloads on origin-a.akamaihd.net. A solution should be in place to forward https to the origin server (eg sniproxy)
new host: #WARNING:Originhasbeenseendownloadinghttpsclientdownloadsonorigin-a.akamaihd.net.Asolutionshouldbeinplacetoforwardhttpstotheoriginserver(egsniproxy)
host: origin-a.akamaihd.net
new host: origin-a.akamaihd.net
host: akamai.cdn.ea.com
new host: akamai.cdn.ea.com
host: lvlt.cdn.ea.com
new host: lvlt.cdn.ea.com
host: river.data.ea.com
new host: river.data.ea.com
host: origin-a.akamaihd.net.edgesuite.net
new host: origin-a.akamaihd.net.edgesuite.net
Reading cache renegadex from renegadex.txt
host: rxp-fl.cncirc.net
new host: rxp-fl.cncirc.net
host: rxp-chi.cncirc.net
new host: rxp-chi.cncirc.net
host: rxp-nz.cncirc.net
new host: rxp-nz.cncirc.net
host: rxp-bgr.cncirc.net
new host: rxp-bgr.cncirc.net
host: rxp-fr.cncirc.net
new host: rxp-fr.cncirc.net
host: rxp-nyc.cncirc.net
new host: rxp-nyc.cncirc.net
host: rxp-uk.cncirc.net
new host: rxp-uk.cncirc.net
host: rxp-sg.cncirc.net
new host: rxp-sg.cncirc.net
host: rxp-la.cncirc.net
new host: rxp-la.cncirc.net
host: rxp-fin.cncirc.net
new host: rxp-fin.cncirc.net
host: denver1.renegade-x.com
new host: denver1.renegade-x.com
Reading cache riot from riot.txt
host: l3cdn.riotgames.com
new host: l3cdn.riotgames.com
host: worldwide.l3cdn.riotgames.com
new host: worldwide.l3cdn.riotgames.com
host: riotgamespatcher-a.akamaihd.net
new host: riotgamespatcher-a.akamaihd.net
host: riotgamespatcher-a.akamaihd.net.edgesuite.net
new host: riotgamespatcher-a.akamaihd.net.edgesuite.net
host: lol.dyn.riotcdn.net
new host: lol.dyn.riotcdn.net
Reading cache rockstar from rockstar.txt
Reading cache sony from sony.txt
host: pls.patch.station.sony.com
new host: pls.patch.station.sony.com
host: gs2.ww.prod.dl.playstation.net
new host: gs2.ww.prod.dl.playstation.net
Reading cache steam from steam.txt
host: *.content.steampowered.com
new host: *.content.steampowered.com
host: content1.steampowered.com
new host: content1.steampowered.com
host: content2.steampowered.com
new host: content2.steampowered.com
host: content3.steampowered.com
new host: content3.steampowered.com
host: content4.steampowered.com
new host: content4.steampowered.com
host: content5.steampowered.com
new host: content5.steampowered.com
host: content6.steampowered.com
new host: content6.steampowered.com
host: content7.steampowered.com
new host: content7.steampowered.com
host: content8.steampowered.com
new host: content8.steampowered.com
host: cs.steampowered.com
new host: cs.steampowered.com
host: steamcontent.com
new host: steamcontent.com
host: client-download.steampowered.com
new host: client-download.steampowered.com
host: *.hsar.steampowered.com.edgesuite.net
new host: *.hsar.steampowered.com.edgesuite.net
host: *.akamai.steamstatic.com
new host: *.akamai.steamstatic.com
host: content-origin.steampowered.com
new host: content-origin.steampowered.com
host: clientconfig.akamai.steamtransparent.com
new host: clientconfig.akamai.steamtransparent.com
host: steampipe.akamaized.net
new host: steampipe.akamaized.net
host: edgecast.steamstatic.com
new host: edgecast.steamstatic.com
host: steam.apac.qtlglb.com.mwcloudcdn.com
new host: steam.apac.qtlglb.com.mwcloudcdn.com
host: *.cs.steampowered.com
new host: *.cs.steampowered.com
host: *.cm.steampowered.com
new host: *.cm.steampowered.com
host: *.edgecast.steamstatic.com
new host: *.edgecast.steamstatic.com
host: *.steamcontent.com
new host: *.steamcontent.com
host: cdn1-sea1.valve.net
new host: cdn1-sea1.valve.net
host: cdn2-sea1.valve.net
new host: cdn2-sea1.valve.net
host: *.steam-content-dnld-1.apac-1-cdn.cqloud.com
new host: *.steam-content-dnld-1.apac-1-cdn.cqloud.com
host: *.steam-content-dnld-1.eu-c1-cdn.cqloud.com
new host: *.steam-content-dnld-1.eu-c1-cdn.cqloud.com
host: steam.apac.qtlglb.com
new host: steam.apac.qtlglb.com
host: edge.steam-dns.top.comcast.net
new host: edge.steam-dns.top.comcast.net
host: edge.steam-dns-2.top.comcast.net
new host: edge.steam-dns-2.top.comcast.net
host: steam.naeu.qtlglb.com
new host: steam.naeu.qtlglb.com
host: steampipe-kr.akamaized.net
new host: steampipe-kr.akamaized.net
host: steam.ix.asn.au
new host: steam.ix.asn.au
host: steam.eca.qtlglb.com
new host: steam.eca.qtlglb.com
host: steam.cdn.on.net
new host: steam.cdn.on.net
host: update5.dota2.wmsj.cn
new host: update5.dota2.wmsj.cn
host: update2.dota2.wmsj.cn
new host: update2.dota2.wmsj.cn
host: update6.dota2.wmsj.cn
new host: update6.dota2.wmsj.cn
host: update3.dota2.wmsj.cn
new host: update3.dota2.wmsj.cn
host: update1.dota2.wmsj.cn
new host: update1.dota2.wmsj.cn
host: update4.dota2.wmsj.cn
new host: update4.dota2.wmsj.cn
host: update5.csgo.wmsj.cn
new host: update5.csgo.wmsj.cn
host: update2.csgo.wmsj.cn
new host: update2.csgo.wmsj.cn
host: update4.csgo.wmsj.cn
new host: update4.csgo.wmsj.cn
host: update3.csgo.wmsj.cn
new host: update3.csgo.wmsj.cn
host: update6.csgo.wmsj.cn
new host: update6.csgo.wmsj.cn
host: update1.csgo.wmsj.cn
new host: update1.csgo.wmsj.cn
host: st.dl.bscstorage.net
new host: st.dl.bscstorage.net
host: cdn.mileweb.cs.steampowered.com.8686c.com
new host: cdn.mileweb.cs.steampowered.com.8686c.com
Reading cache uplay from uplay.txt
host: *.cdn.ubi.com
new host: *.cdn.ubi.com
Reading cache twitch from twitchapp.txt
host: d3rmjivj4k4f0t.cloudfront.net
new host: d3rmjivj4k4f0t.cloudfront.net
host: addons.forgesvc.net
new host: addons.forgesvc.net
host: media.forgecdn.net
new host: media.forgecdn.net
host: files.forgecdn.net
new host: files.forgecdn.net
Reading cache wargaming from wargaming.net.txt
host: dl1.wargaming.net
new host: dl1.wargaming.net
host: dl2.wargaming.net
new host: dl2.wargaming.net
host: wg.gcdn.co
new host: wg.gcdn.co
host: wgusst-na.wargaming.net
new host: wgusst-na.wargaming.net
host: wgusst-eu.wargaming.net
new host: wgusst-eu.wargaming.net
host: update-v4r4h10x.worldofwarships.com
new host: update-v4r4h10x.worldofwarships.com
host: wgus-wotasia.wargaming.net
new host: wgus-wotasia.wargaming.net
host: dl-wot-ak.wargaming.net
new host: dl-wot-ak.wargaming.net
host: dl-wot-gc.wargaming.net
new host: dl-wot-gc.wargaming.net
host: dl-wot-se.wargaming.net
new host: dl-wot-se.wargaming.net
host: dl-wot-cdx.wargaming.net
new host: dl-wot-cdx.wargaming.net
host: dl-wows-ak.wargaming.net
new host: dl-wows-ak.wargaming.net
host: dl-wows-gc.wargaming.net
new host: dl-wows-gc.wargaming.net
host: dl-wows-se.wargaming.net
new host: dl-wows-se.wargaming.net
host: dl-wows-cdx.wargaming.net
new host: dl-wows-cdx.wargaming.net
host: dl-wowp-ak.wargaming.net
new host: dl-wowp-ak.wargaming.net
host: dl-wowp-gc.wargaming.net
new host: dl-wowp-gc.wargaming.net
host: dl-wowp-se.wargaming.net
new host: dl-wowp-se.wargaming.net
host: dl-wowp-cdx.wargaming.net
new host: dl-wowp-cdx.wargaming.net
Reading cache wsus from windowsupdates.txt
host: officecdn.microsoft.com
new host: officecdn.microsoft.com
host: *.windowsupdate.com
new host: *.windowsupdate.com
host: windowsupdate.com
new host: windowsupdate.com
host: *.dl.delivery.mp.microsoft.com
new host: *.dl.delivery.mp.microsoft.com
host: dl.delivery.mp.microsoft.com
new host: dl.delivery.mp.microsoft.com
host: *.update.microsoft.com
new host: *.update.microsoft.com
host: *.do.dsp.mp.microsoft.com
new host: *.do.dsp.mp.microsoft.com
host: *.microsoft.com.edgesuite.net
new host: *.microsoft.com.edgesuite.net
Reading cache xboxlive from xboxlive.txt
host: assets1.xboxlive.com
new host: assets1.xboxlive.com
host: assets2.xboxlive.com
new host: assets2.xboxlive.com
host: dlassets.xboxlive.com
new host: dlassets.xboxlive.com
host: xboxone.loris.llnwd.net
new host: xboxone.loris.llnwd.net
host: *.xboxone.loris.llnwd.net
new host: *.xboxone.loris.llnwd.net
host: xboxone.vo.llnwd.net
new host: xboxone.vo.llnwd.net
host: xbox-mbr.xboxlive.com
new host: xbox-mbr.xboxlive.com
host: assets1.xboxlive.com.nsatc.net
new host: assets1.xboxlive.com.nsatc.net
map $http_host $cacheidentifier {
hostnames;
default $http_host;
assetcdn.101.arenanetworks.com arenanet;
assetcdn.102.arenanetworks.com arenanet;
assetcdn.103.arenanetworks.com arenanet;
dist.blizzard.com blizzard;
dist.blizzard.com.edgesuite.net blizzard;
llnw.blizzard.com blizzard;
edgecast.blizzard.com blizzard;
blizzard.vo.llnwd.net blizzard;
blzddist1-a.akamaihd.net blizzard;
blzddist2-a.akamaihd.net blizzard;
blzddist3-a.akamaihd.net blizzard;
blzddist4-a.akamaihd.net blizzard;
level3.blizzard.com blizzard;
nydus.battle.net blizzard;
edge.blizzard.top.comcast.net blizzard;
cdn.blizzard.com blizzard;
*.cdn.blizzard.com blizzard;
#Daybreakgames daybreak;
# daybreak;
#DoNOTcachemanifest.patch.daybreakgames.com daybreak;
# daybreak;
#PS2 daybreak;
pls.patch.daybreakgames.com daybreak;
epicgames-download1.akamaized.net epicgames;
download.epicgames.com epicgames;
download2.epicgames.com epicgames;
download3.epicgames.com epicgames;
download4.epicgames.com epicgames;
ccs.cdn.wup.shop.nintendo.com nintendo;
pushmo.hac.lp1.eshop.nintendo.net nintendo;
ecs-lp1.hac.shop.nintendo.net nintendo;
receive-lp1.dg.srv.nintendo.net nintendo;
aqua.hac.lp1.d4c.nintendo.net nintendo;
atum.hac.lp1.d4c.nintendo.net nintendo;
bugyo.hac.lp1.eshop.nintendo.net nintendo;
tagaya.hac.lp1.eshop.nintendo.net nintendo;
#WARNING:Originhasbeenseendownloadinghttpsclientdownloadsonorigin-a.akamaihd.net.Asolutionshouldbeinplacetoforwardhttpstotheoriginserver(egsniproxy) origin;
origin-a.akamaihd.net origin;
akamai.cdn.ea.com origin;
lvlt.cdn.ea.com origin;
river.data.ea.com origin;
origin-a.akamaihd.net.edgesuite.net origin;
rxp-fl.cncirc.net renegadex;
rxp-chi.cncirc.net renegadex;
rxp-nz.cncirc.net renegadex;
rxp-bgr.cncirc.net renegadex;
rxp-fr.cncirc.net renegadex;
rxp-nyc.cncirc.net renegadex;
rxp-uk.cncirc.net renegadex;
rxp-sg.cncirc.net renegadex;
rxp-la.cncirc.net renegadex;
rxp-fin.cncirc.net renegadex;
denver1.renegade-x.com renegadex;
l3cdn.riotgames.com riot;
worldwide.l3cdn.riotgames.com riot;
riotgamespatcher-a.akamaihd.net riot;
riotgamespatcher-a.akamaihd.net.edgesuite.net riot;
lol.dyn.riotcdn.net riot;
pls.patch.station.sony.com sony;
gs2.ww.prod.dl.playstation.net sony;
*.content.steampowered.com steam;
content1.steampowered.com steam;
content2.steampowered.com steam;
content3.steampowered.com steam;
content4.steampowered.com steam;
content5.steampowered.com steam;
content6.steampowered.com steam;
content7.steampowered.com steam;
content8.steampowered.com steam;
cs.steampowered.com steam;
steamcontent.com steam;
client-download.steampowered.com steam;
*.hsar.steampowered.com.edgesuite.net steam;
*.akamai.steamstatic.com steam;
content-origin.steampowered.com steam;
clientconfig.akamai.steamtransparent.com steam;
steampipe.akamaized.net steam;
edgecast.steamstatic.com steam;
steam.apac.qtlglb.com.mwcloudcdn.com steam;
*.cs.steampowered.com steam;
*.cm.steampowered.com steam;
*.edgecast.steamstatic.com steam;
*.steamcontent.com steam;
cdn1-sea1.valve.net steam;
cdn2-sea1.valve.net steam;
*.steam-content-dnld-1.apac-1-cdn.cqloud.com steam;
*.steam-content-dnld-1.eu-c1-cdn.cqloud.com steam;
steam.apac.qtlglb.com steam;
edge.steam-dns.top.comcast.net steam;
edge.steam-dns-2.top.comcast.net steam;
steam.naeu.qtlglb.com steam;
steampipe-kr.akamaized.net steam;
steam.ix.asn.au steam;
steam.eca.qtlglb.com steam;
steam.cdn.on.net steam;
update5.dota2.wmsj.cn steam;
update2.dota2.wmsj.cn steam;
update6.dota2.wmsj.cn steam;
update3.dota2.wmsj.cn steam;
update1.dota2.wmsj.cn steam;
update4.dota2.wmsj.cn steam;
update5.csgo.wmsj.cn steam;
update2.csgo.wmsj.cn steam;
update4.csgo.wmsj.cn steam;
update3.csgo.wmsj.cn steam;
update6.csgo.wmsj.cn steam;
update1.csgo.wmsj.cn steam;
st.dl.bscstorage.net steam;
cdn.mileweb.cs.steampowered.com.8686c.com steam;
*.cdn.ubi.com uplay;
d3rmjivj4k4f0t.cloudfront.net twitch;
addons.forgesvc.net twitch;
media.forgecdn.net twitch;
files.forgecdn.net twitch;
dl1.wargaming.net wargaming;
dl2.wargaming.net wargaming;
wg.gcdn.co wargaming;
wgusst-na.wargaming.net wargaming;
wgusst-eu.wargaming.net wargaming;
update-v4r4h10x.worldofwarships.com wargaming;
wgus-wotasia.wargaming.net wargaming;
dl-wot-ak.wargaming.net wargaming;
dl-wot-gc.wargaming.net wargaming;
dl-wot-se.wargaming.net wargaming;
dl-wot-cdx.wargaming.net wargaming;
dl-wows-ak.wargaming.net wargaming;
dl-wows-gc.wargaming.net wargaming;
dl-wows-se.wargaming.net wargaming;
dl-wows-cdx.wargaming.net wargaming;
dl-wowp-ak.wargaming.net wargaming;
dl-wowp-gc.wargaming.net wargaming;
dl-wowp-se.wargaming.net wargaming;
dl-wowp-cdx.wargaming.net wargaming;
officecdn.microsoft.com wsus;
*.windowsupdate.com wsus;
windowsupdate.com wsus;
*.dl.delivery.mp.microsoft.com wsus;
dl.delivery.mp.microsoft.com wsus;
*.update.microsoft.com wsus;
*.do.dsp.mp.microsoft.com wsus;
*.microsoft.com.edgesuite.net wsus;
assets1.xboxlive.com xboxlive;
assets2.xboxlive.com xboxlive;
dlassets.xboxlive.com xboxlive;
xboxone.loris.llnwd.net xboxlive;
*.xboxone.loris.llnwd.net xboxlive;
xboxone.vo.llnwd.net xboxlive;
xbox-mbr.xboxlive.com xboxlive;
assets1.xboxlive.com.nsatc.net xboxlive;
}
Executing hook /hooks/entrypoint-pre.d/19_doc_root_setup
/var/www/html already exists.
Setting document root to /var/www/html
Executing hook /hooks/entrypoint-pre.d/20_perms_check.sh
Executing hook /hooks/entrypoint-pre.d/20_ssl_setup
Not enabling SSL as neither key nor cert provided.
Executing hook /hooks/supervisord-pre.d/20_test_files_setup
Checking if /var/www/html is empty - Directory not empty.. don't touch content
Executing hook /hooks/supervisord-pre.d/21_cleanup_log_files
Cleaning up log files older than 3560 days
Executing hook /hooks/supervisord-pre.d/99_config_check.sh
Currently configured config:
NGINX CONFIG DUMP FOR /etc/nginx/nginx.conf
# Including /etc/nginx/nginx.conf
user www-data;
# Including /etc/nginx/workers.conf
worker_processes auto;
# Finished including /etc/nginx/workers.conf
pid /run/nginx.pid;
events {
worker_connections 4096;
multi_accept on;
use epoll;
}
http {
aio threads;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
# Including /etc/nginx/mime.types
types {
text/html html htm shtml;
text/css css;
text/xml xml;
image/gif gif;
image/jpeg jpeg jpg;
application/javascript js;
application/atom+xml atom;
application/rss+xml rss;
text/mathml mml;
text/plain txt;
text/vnd.sun.j2me.app-descriptor jad;
text/vnd.wap.wml wml;
text/x-component htc;
image/png png;
image/tiff tif tiff;
image/vnd.wap.wbmp wbmp;
image/x-icon ico;
image/x-jng jng;
image/x-ms-bmp bmp;
image/svg+xml svg svgz;
image/webp webp;
application/font-woff woff;
application/java-archive jar war ear;
application/json json;
application/mac-binhex40 hqx;
application/msword doc;
application/pdf pdf;
application/postscript ps eps ai;
application/rtf rtf;
application/vnd.apple.mpegurl m3u8;
application/vnd.ms-excel xls;
application/vnd.ms-fontobject eot;
application/vnd.ms-powerpoint ppt;
application/vnd.wap.wmlc wmlc;
application/vnd.google-earth.kml+xml kml;
application/vnd.google-earth.kmz kmz;
application/x-7z-compressed 7z;
application/x-cocoa cco;
application/x-java-archive-diff jardiff;
application/x-java-jnlp-file jnlp;
application/x-makeself run;
application/x-perl pl pm;
application/x-pilot prc pdb;
application/x-rar-compressed rar;
application/x-redhat-package-manager rpm;
application/x-sea sea;
application/x-shockwave-flash swf;
application/x-stuffit sit;
application/x-tcl tcl tk;
application/x-x509-ca-cert der pem crt;
application/x-xpinstall xpi;
application/xhtml+xml xhtml;
application/xspf+xml xspf;
application/zip zip;
application/octet-stream bin exe dll;
application/octet-stream deb;
application/octet-stream dmg;
application/octet-stream iso img;
application/octet-stream msi msp msm;
application/vnd.openxmlformats-officedocument.wordprocessingml.document docx;
application/vnd.openxmlformats-officedocument.spreadsheetml.sheet xlsx;
application/vnd.openxmlformats-officedocument.presentationml.presentation pptx;
audio/midi mid midi kar;
audio/mpeg mp3;
audio/ogg ogg;
audio/x-m4a m4a;
audio/x-realaudio ra;
video/3gpp 3gpp 3gp;
video/mp2t ts;
video/mp4 mp4;
video/mpeg mpeg mpg;
video/quicktime mov;
video/webm webm;
video/x-flv flv;
video/x-m4v m4v;
video/x-mng mng;
video/x-ms-asf asx asf;
video/x-ms-wmv wmv;
video/x-msvideo avi;
}
# Finished including /etc/nginx/mime.types
default_type application/octet-stream;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
gzip on;
# Including /etc/nginx/conf.d/10_log_format.conf
log_format cachelog '[$cacheidentifier] $remote_addr / $http_x_forwarded_for - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" "$upstream_cache_status" "$host" "$http_range"';
# Finished including /etc/nginx/conf.d/10_log_format.conf
# Including /etc/nginx/conf.d/20_proxy_cache_path.conf
proxy_cache_path /data/cache/cache levels=2:2 keys_zone=generic:500m inactive=200d max_size=1000000m loader_files=1000 loader_sleep=50ms loader_threshold=300ms use_temp_path=off;
# Finished including /etc/nginx/conf.d/20_proxy_cache_path.conf
# Including /etc/nginx/conf.d/30_maps.conf
map $http_host $cacheidentifier {
hostnames;
default $http_host;
assetcdn.101.arenanetworks.com arenanet;
assetcdn.102.arenanetworks.com arenanet;
assetcdn.103.arenanetworks.com arenanet;
dist.blizzard.com blizzard;
dist.blizzard.com.edgesuite.net blizzard;
llnw.blizzard.com blizzard;
edgecast.blizzard.com blizzard;
blizzard.vo.llnwd.net blizzard;
blzddist1-a.akamaihd.net blizzard;
blzddist2-a.akamaihd.net blizzard;
blzddist3-a.akamaihd.net blizzard;
blzddist4-a.akamaihd.net blizzard;
level3.blizzard.com blizzard;
nydus.battle.net blizzard;
edge.blizzard.top.comcast.net blizzard;
cdn.blizzard.com blizzard;
*.cdn.blizzard.com blizzard;
#Daybreakgames daybreak;
# daybreak;
#DoNOTcachemanifest.patch.daybreakgames.com daybreak;
# daybreak;
#PS2 daybreak;
pls.patch.daybreakgames.com daybreak;
epicgames-download1.akamaized.net epicgames;
download.epicgames.com epicgames;
download2.epicgames.com epicgames;
download3.epicgames.com epicgames;
download4.epicgames.com epicgames;
ccs.cdn.wup.shop.nintendo.com nintendo;
pushmo.hac.lp1.eshop.nintendo.net nintendo;
ecs-lp1.hac.shop.nintendo.net nintendo;
receive-lp1.dg.srv.nintendo.net nintendo;
aqua.hac.lp1.d4c.nintendo.net nintendo;
atum.hac.lp1.d4c.nintendo.net nintendo;
bugyo.hac.lp1.eshop.nintendo.net nintendo;
tagaya.hac.lp1.eshop.nintendo.net nintendo;
#WARNING:Originhasbeenseendownloadinghttpsclientdownloadsonorigin-a.akamaihd.net.Asolutionshouldbeinplacetoforwardhttpstotheoriginserver(egsniproxy) origin;
origin-a.akamaihd.net origin;
akamai.cdn.ea.com origin;
lvlt.cdn.ea.com origin;
river.data.ea.com origin;
origin-a.akamaihd.net.edgesuite.net origin;
rxp-fl.cncirc.net renegadex;
rxp-chi.cncirc.net renegadex;
rxp-nz.cncirc.net renegadex;
rxp-bgr.cncirc.net renegadex;
rxp-fr.cncirc.net renegadex;
rxp-nyc.cncirc.net renegadex;
rxp-uk.cncirc.net renegadex;
rxp-sg.cncirc.net renegadex;
rxp-la.cncirc.net renegadex;
rxp-fin.cncirc.net renegadex;
denver1.renegade-x.com renegadex;
l3cdn.riotgames.com riot;
worldwide.l3cdn.riotgames.com riot;
riotgamespatcher-a.akamaihd.net riot;
riotgamespatcher-a.akamaihd.net.edgesuite.net riot;
lol.dyn.riotcdn.net riot;
pls.patch.station.sony.com sony;
gs2.ww.prod.dl.playstation.net sony;
*.content.steampowered.com steam;
content1.steampowered.com steam;
content2.steampowered.com steam;
content3.steampowered.com steam;
content4.steampowered.com steam;
content5.steampowered.com steam;
content6.steampowered.com steam;
content7.steampowered.com steam;
content8.steampowered.com steam;
cs.steampowered.com steam;
steamcontent.com steam;
client-download.steampowered.com steam;
*.hsar.steampowered.com.edgesuite.net steam;
*.akamai.steamstatic.com steam;
content-origin.steampowered.com steam;
clientconfig.akamai.steamtransparent.com steam;
steampipe.akamaized.net steam;
edgecast.steamstatic.com steam;
steam.apac.qtlglb.com.mwcloudcdn.com steam;
*.cs.steampowered.com steam;
*.cm.steampowered.com steam;
*.edgecast.steamstatic.com steam;
*.steamcontent.com steam;
cdn1-sea1.valve.net steam;
cdn2-sea1.valve.net steam;
*.steam-content-dnld-1.apac-1-cdn.cqloud.com steam;
*.steam-content-dnld-1.eu-c1-cdn.cqloud.com steam;
steam.apac.qtlglb.com steam;
edge.steam-dns.top.comcast.net steam;
edge.steam-dns-2.top.comcast.net steam;
steam.naeu.qtlglb.com steam;
steampipe-kr.akamaized.net steam;
steam.ix.asn.au steam;
steam.eca.qtlglb.com steam;
steam.cdn.on.net steam;
update5.dota2.wmsj.cn steam;
update2.dota2.wmsj.cn steam;
update6.dota2.wmsj.cn steam;
update3.dota2.wmsj.cn steam;
update1.dota2.wmsj.cn steam;
update4.dota2.wmsj.cn steam;
update5.csgo.wmsj.cn steam;
update2.csgo.wmsj.cn steam;
update4.csgo.wmsj.cn steam;
update3.csgo.wmsj.cn steam;
update6.csgo.wmsj.cn steam;
update1.csgo.wmsj.cn steam;
st.dl.bscstorage.net steam;
cdn.mileweb.cs.steampowered.com.8686c.com steam;
*.cdn.ubi.com uplay;
d3rmjivj4k4f0t.cloudfront.net twitch;
addons.forgesvc.net twitch;
media.forgecdn.net twitch;
files.forgecdn.net twitch;
dl1.wargaming.net wargaming;
dl2.wargaming.net wargaming;
wg.gcdn.co wargaming;
wgusst-na.wargaming.net wargaming;
wgusst-eu.wargaming.net wargaming;
update-v4r4h10x.worldofwarships.com wargaming;
wgus-wotasia.wargaming.net wargaming;
dl-wot-ak.wargaming.net wargaming;
dl-wot-gc.wargaming.net wargaming;
dl-wot-se.wargaming.net wargaming;
dl-wot-cdx.wargaming.net wargaming;
dl-wows-ak.wargaming.net wargaming;
dl-wows-gc.wargaming.net wargaming;
dl-wows-se.wargaming.net wargaming;
dl-wows-cdx.wargaming.net wargaming;
dl-wowp-ak.wargaming.net wargaming;
dl-wowp-gc.wargaming.net wargaming;
dl-wowp-se.wargaming.net wargaming;
dl-wowp-cdx.wargaming.net wargaming;
officecdn.microsoft.com wsus;
*.windowsupdate.com wsus;
windowsupdate.com wsus;
*.dl.delivery.mp.microsoft.com wsus;
dl.delivery.mp.microsoft.com wsus;
*.update.microsoft.com wsus;
*.do.dsp.mp.microsoft.com wsus;
*.microsoft.com.edgesuite.net wsus;
assets1.xboxlive.com xboxlive;
assets2.xboxlive.com xboxlive;
dlassets.xboxlive.com xboxlive;
xboxone.loris.llnwd.net xboxlive;
*.xboxone.loris.llnwd.net xboxlive;
xboxone.vo.llnwd.net xboxlive;
xbox-mbr.xboxlive.com xboxlive;
assets1.xboxlive.com.nsatc.net xboxlive;
}
# Finished including /etc/nginx/conf.d/30_maps.conf
# Including /etc/nginx/sites-enabled/10_generic.conf
server {
listen 80 reuseport;
access_log /data/logs/access.log cachelog;
error_log /data/logs/error.log;
# Including /etc/nginx/sites-available/generic.conf.d/10_generic.conf
resolver 8.8.8.8 8.8.4.4 ipv6=off;
location / {
# Including /etc/nginx/sites-available/generic.conf.d/root/10_loop_detection.conf
# Abort any circular requests
if ($http_X_LanCache_Processed_By = $hostname) {
return 508;
}
proxy_set_header X-LanCache-Processed-By $hostname;
add_header X-LanCache-Processed-By $hostname,$http_X_LanCache_Processed_By;
# Finished including /etc/nginx/sites-available/generic.conf.d/root/10_loop_detection.conf
# Including /etc/nginx/sites-available/generic.conf.d/root/20_cache.conf
# Cache Location
slice 1m;
proxy_cache generic;
proxy_ignore_headers Expires Cache-Control;
proxy_cache_valid 200 206 3560d;
proxy_set_header Range $slice_range;
# Only download one copy at a time and use a large timeout so
# this really happens, otherwise we end up wasting bandwith
# getting the file multiple times.
proxy_cache_lock on;
proxy_cache_lock_timeout 1h;
# Allow the use of state entries
proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;
# Allow caching of 200 but not 301 or 302 as our cache key may not include query params
# hence may not be valid for all users
proxy_cache_valid 301 302 0;
# Enable cache revalidation
proxy_cache_revalidate on;
# Don't cache requests marked as nocache=1
proxy_cache_bypass $arg_nocache;
# 40G max file
proxy_max_temp_file_size 40960m;
# Finished including /etc/nginx/sites-available/generic.conf.d/root/20_cache.conf
# Including /etc/nginx/sites-available/generic.conf.d/root/30_cache_key.conf
proxy_cache_key $cacheidentifier$uri$slice_range;
# Finished including /etc/nginx/sites-available/generic.conf.d/root/30_cache_key.conf
# Including /etc/nginx/sites-available/generic.conf.d/root/40_etags.conf
# Battle.net Fix
proxy_hide_header ETag;
# Finished including /etc/nginx/sites-available/generic.conf.d/root/40_etags.conf
# Including /etc/nginx/sites-available/generic.conf.d/root/90_upstream.conf
# Upstream Configuration
proxy_next_upstream error timeout http_404;
proxy_pass http://$host$request_uri;
proxy_redirect off;
proxy_ignore_client_abort on;
# Upstream request headers
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# Finished including /etc/nginx/sites-available/generic.conf.d/root/90_upstream.conf
# Including /etc/nginx/sites-available/generic.conf.d/root/99_debug_header.conf
# Debug Headers
add_header X-Upstream-Status $upstream_status;
add_header X-Upstream-Response-Time $upstream_response_time;
add_header X-Upstream-Cache-Status $upstream_cache_status;
# Finished including /etc/nginx/sites-available/generic.conf.d/root/99_debug_header.conf
}
# Fix for League of Legends Updater
location ~ ^.+(releaselisting_.*|.version$) {
proxy_pass http://$host;
}
location = /steamcache-heartbeat {
add_header X-LanCache-Processed-By $hostname;
return 204;
}
# Finished including /etc/nginx/sites-available/generic.conf.d/10_generic.conf
}
# Finished including /etc/nginx/sites-enabled/10_generic.conf
}
# Finished including /etc/nginx/nginx.conf
Checking nginx config
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: [emerg] mkdir() "/data/cache/cache" failed (13: Permission denied)
nginx: configuration file /etc/nginx/nginx.conf test failed
ERROR: hook /hooks/supervisord-pre.d/99_config_check.sh} returned a non-zero exit status '0'
Update handling
I feel weird asking this in the format of an issue, but that is what the README informed me to do if I did not find the information I was looking for in the FAQ. Additionally I was unable to find this information online via search or in previous “issues”.
My question is quite simple hopefully! If a game was already cached on the server, and then an update for the game was released before one of the computers started downloading the cached version, would the entire game need to be re-downloaded? Or would it download the cached version from the cache, then download the update from the CDN and cache the update? Or something else entirely?
Thank you for your time, and if this question has already been answered before, I apologize deeply.
[Tracking] steamcache not caching steam requests due to HTTPS
Describe the issue you are having
Downloading Steam games does not write cache data to the disk. I can see the traffic is going through my lancache server, but nothing is being written to the disk. Strangely enough, this only happens to Steam. Caching Uplay and Microsoft work just fine.
How are you running the container(s)?
docker run --detach \
--restart unless-stopped \
--name steamcache-dns \
-p 192.168.1.10:53:53/udp \
-e USE_GENERIC_CACHE=true \
-e LANCACHE_IP=192.168.1.10 \
-e UPSTREAM_DNS=192.168.1.1 \
steamcache/steamcache-dns:latest
docker run --detach \
--restart unless-stopped \
--name lancache \
-v $HOME/lancache-data/cache:/data/cache \
-v $HOME/lancache-data/logs:/data/logs \
-p 80:80 \
-e CACHE_MEM_SIZE=512m \
-e CACHE_DISK_SIZE=180g \
steamcache/monolithic:latest
docker run --detach \
--restart unless-stopped \
--name sniproxy \
-p 443:443 \
steamcache/sniproxy:latest
DNS Configuration
UPSTREAM_DNS is pointed to my router, which is redirected to a VM running pi-hole. I suspected this might be the reason, but I was wrong. The same problem persists even when I set UPSTREAM_DNS to Google DNS.
Output of container(s)
$ docker exec -it lancache tail -f /data/logs/access.log
[uplay] 192.168.1.174 / - - - [27/Apr/2019:06:14:55 +0000] "GET /uplaypc/downloads/720/slices/7B5A972296D93C2133FA8C85EFCE91B5A26FED25?_tkn_=exp=1556349271~acl=%2fuplaypc%2fdownloads%2f720%2fslices%2f7B5A972296D93C2133FA8C85EFCE91B5A26FED25~data=2e2896c5-91f8-4451-8bb8-80e24eb51b39~hmac=5801f66a896378f8ddd707a5189796701a80e290337d445287373909e5a82dfd HTTP/1.1" 200 2097152 "-" "Massgate" "MISS" "uplaypc-s-ubisoft.cdn.ubi.com" "-"
[uplay] 192.168.1.174 / - - - [27/Apr/2019:06:14:56 +0000] "GET /uplaypc/downloads/720/slices/4EFF1460A3AEE2D5E65A9DA7A38EE0958E760C5E?_tkn_=exp=1556349267~acl=%2fuplaypc%2fdownloads%2f720%2fslices%2f4EFF1460A3AEE2D5E65A9DA7A38EE0958E760C5E~data=2e2896c5-91f8-4451-8bb8-80e24eb51b39~hmac=d673b1cd71e26f20429bde9aa2a9816fb14c450139838ce9b78feeb90658f356 HTTP/1.1" 200 1256701 "-" "Massgate" "MISS" "uplaypc-s-ubisoft.cdn.ubi.com" "-"
[uplay] 192.168.1.174 / - - - [27/Apr/2019:06:14:56 +0000] "GET /uplaypc/downloads/720/slices/86B18AF05CE886B8D9F1B3174517BA2EF6C2D953?_tkn_=exp=1556349266~acl=%2fuplaypc%2fdownloads%2f720%2fslices%2f86B18AF05CE886B8D9F1B3174517BA2EF6C2D953~data=2e2896c5-91f8-4451-8bb8-80e24eb51b39~hmac=fbdfed378cf04fb62bc24f01c41dd3488a03266d46240c1ccc615efb3e84d835 HTTP/1.1" 200 1236221 "-" "Massgate" "MISS" "uplaypc-s-ubisoft.cdn.ubi.com" "-"
[uplay] 192.168.1.174 / - - - [27/Apr/2019:06:14:56 +0000] "GET /uplaypc/downloads/720/slices/6B04FC3FB1DED37559DEF38735091881317BADAB?_tkn_=exp=1556349273~acl=%2fuplaypc%2fdownloads%2f720%2fslices%2f6B04FC3FB1DED37559DEF38735091881317BADAB~data=2e2896c5-91f8-4451-8bb8-80e24eb51b39~hmac=9e31c87cf69f7a88b4abd9c64264bbc9e192f27978dc4a45f2709d43e75c763f HTTP/1.1" 200 670985 "-" "Massgate" "MISS" "uplaypc-s-ubisoft.cdn.ubi.com" "-"
[uplay] 192.168.1.174 / - - - [27/Apr/2019:06:14:56 +0000] "GET /uplaypc/downloads/720/slices/A74C9A8C2CE6364B5259D1556027DE82832EF3B8?_tkn_=exp=1556349271~acl=%2fuplaypc%2fdownloads%2f720%2fslices%2fA74C9A8C2CE6364B5259D1556027DE82832EF3B8~data=2e2896c5-91f8-4451-8bb8-80e24eb51b39~hmac=0061c05a81078c9ead96618d3370617b46c7d728ae6e666b86283e739c50228e HTTP/1.1" 200 1715453 "-" "Massgate" "MISS" "uplaypc-s-ubisoft.cdn.ubi.com" "-"
[uplay] 192.168.1.174 / - - - [27/Apr/2019:06:14:56 +0000] "GET /uplaypc/downloads/720/slices/BC5E963BC134EC261B6FEB47B44FC60EB8A2BC4E?_tkn_=exp=1556349267~acl=%2fuplaypc%2fdownloads%2f720%2fslices%2fBC5E963BC134EC261B6FEB47B44FC60EB8A2BC4E~data=2e2896c5-91f8-4451-8bb8-80e24eb51b39~hmac=28b8671eef4f1c3f340c4e6959e464baf8fff33ff1a0043dc36750280cc52b09 HTTP/1.1" 200 1649917 "-" "Massgate" "MISS" "uplaypc-s-ubisoft.cdn.ubi.com" "-"
[uplay] 192.168.1.174 / - - - [27/Apr/2019:06:14:56 +0000] "GET /uplaypc/downloads/720/slices/F1FD766104F2F807F7C0308501FA58DDCB565F97?_tkn_=exp=1556349273~acl=%2fuplaypc%2fdownloads%2f720%2fslices%2fF1FD766104F2F807F7C0308501FA58DDCB565F97~data=2e2896c5-91f8-4451-8bb8-80e24eb51b39~hmac=cbc563d1fb72590248b8881aa98d0187ca6027a985431ce7599abf692768293a HTTP/1.1" 200 720137 "-" "Massgate" "MISS" "uplaypc-s-ubisoft.cdn.ubi.com" "-"
[uplay] 192.168.1.174 / - - - [27/Apr/2019:06:14:56 +0000] "GET /uplaypc/downloads/720/slices/EC8FBF7A0D6E9EF8692E46DDF251331C91652CB5?_tkn_=exp=1556349272~acl=%2fuplaypc%2fdownloads%2f720%2fslices%2fEC8FBF7A0D6E9EF8692E46DDF251331C91652CB5~data=2e2896c5-91f8-4451-8bb8-80e24eb51b39~hmac=8c5499fe882b3fe8c76c2c2f2f8179f6c0d1c3b208d6734ae9a33616cd396940 HTTP/1.1" 200 691465 "-" "Massgate" "MISS" "uplaypc-s-ubisoft.cdn.ubi.com" "-"
[uplay] 192.168.1.174 / - - - [27/Apr/2019:06:14:56 +0000] "GET /uplaypc/downloads/720/slices/41AC96F16F89C71F750D19808332B97C54E5A0FA?_tkn_=exp=1556349272~acl=%2fuplaypc%2fdownloads%2f720%2fslices%2f41AC96F16F89C71F750D19808332B97C54E5A0FA~data=2e2896c5-91f8-4451-8bb8-80e24eb51b39~hmac=8db46d446aa8136dc932016f75c82b6741c27f58a8078027c3599324575b0f29 HTTP/1.1" 200 1064189 "-" "Massgate" "MISS" "uplaypc-s-ubisoft.cdn.ubi.com" "-"
[steam] 192.168.1.174 / - - - [27/Apr/2019:06:15:04 +0000] "GET /client/steam_client_win32 HTTP/1.1" 302 0 "-" "Valve/Steam HTTP Client 1.0 (client;windows;16;1555457005)" "MISS" "client-download.steampowered.com" "-"
I expected new lines when downlowing Steam games, but there was none.
Tried to change environment variables and now it won't start
Describe the issue you are having
Tried to change environment variables and now it won't start
Output of container(s)
Executing hook /hooks/entrypoint-pre.d/19_doc_root_setup
/var/www/html already exists.
Setting document root to /var/www/html
Executing hook /hooks/entrypoint-pre.d/20_perms_check.sh
Checking permissions (This may take a long time if the permissions are incorrect on large caches)...
Permisions ok
Executing hook /hooks/entrypoint-pre.d/20_ssl_setup
Not enabling SSL as neither key nor cert provided.
/init/entrypoint: line 66: /scripts/'/init/supervisord': No such file or directory
Executing hook /hooks/entrypoint-pre.d/10_setup.sh
Executing hook /hooks/entrypoint-pre.d/15_generate_maps.sh
HEAD is now at db4f1f8 Remove images-eds from xboxlive (#50)
Reading cache apple from apple.txt
Reading cache arenanet from arenanet.txt
Reading cache blizzard from blizzard.txt
Reading cache daybreak from daybreak.txt
Reading cache frontier from frontier.txt
Reading cache hirez from hirez.txt
Reading cache nexusmods from nexusmods.txt
Reading cache nintendo from nintendo.txt
Reading cache origin from origin.txt
Reading cache renegadex from renegadex.txt
Reading cache riot from riot.txt
Reading cache rockstar from rockstar.txt
Reading cache sony from sony.txt
Reading cache steam from steam.txt
Reading cache uplay from uplay.txt
Reading cache twitch from twitchapp.txt
Reading cache wargaming from wargaming.net.txt
Reading cache wsus from windowsupdates.txt
Reading cache xboxlive from xboxlive.txt
map $http_host $cacheidentifier {
hostnames;
default $http_host;
assetcdn.101.arenanetworks.com arenanet;
assetcdn.102.arenanetworks.com arenanet;
assetcdn.103.arenanetworks.com arenanet;
dist.blizzard.com blizzard;
dist.blizzard.com.edgesuite.net blizzard;
llnw.blizzard.com blizzard;
edgecast.blizzard.com blizzard;
blizzard.vo.llnwd.net blizzard;
blzddist1-a.akamaihd.net blizzard;
blzddist2-a.akamaihd.net blizzard;
blzddist3-a.akamaihd.net blizzard;
blzddist4-a.akamaihd.net blizzard;
level3.blizzard.com blizzard;
nydus.battle.net blizzard;
edge.blizzard.top.comcast.net blizzard;
cdn.blizzard.com blizzard;
*.cdn.blizzard.com blizzard;
# Daybreak games daybreak;
# daybreak;
# Do NOT cache manifest.patch.daybreakgames.com daybreak;
# daybreak;
#PS2 daybreak;
pls.patch.daybreakgames.com daybreak;
ccs.cdn.wup.shop.nintendo.com nintendo;
pushmo.hac.lp1.eshop.nintendo.net nintendo;
ecs-lp1.hac.shop.nintendo.net nintendo;
receive-lp1.dg.srv.nintendo.net nintendo;
aqua.hac.lp1.d4c.nintendo.net nintendo;
atum.hac.lp1.d4c.nintendo.net nintendo;
bugyo.hac.lp1.eshop.nintendo.net nintendo;
tagaya.hac.lp1.eshop.nintendo.net nintendo;
# WARNING: Origin has been seen downloading https client downloads on origin-a.akamaihd.net. A solution should be in place to forward https to the origin server (eg sniproxy) origin;
origin-a.akamaihd.net origin;
akamai.cdn.ea.com origin;
lvlt.cdn.ea.com origin;
river.data.ea.com origin;
origin-a.akamaihd.net.edgesuite.net origin;
rxp-fl.cncirc.net renegadex;
rxp-chi.cncirc.net renegadex;
rxp-nz.cncirc.net renegadex;
rxp-bgr.cncirc.net renegadex;
rxp-fr.cncirc.net renegadex;
rxp-nyc.cncirc.net renegadex;
rxp-uk.cncirc.net renegadex;
rxp-sg.cncirc.net renegadex;
rxp-la.cncirc.net renegadex;
rxp-fin.cncirc.net renegadex;
denver1.renegade-x.com renegadex;
l3cdn.riotgames.com riot;
worldwide.l3cdn.riotgames.com riot;
riotgamespatcher-a.akamaihd.net riot;
riotgamespatcher-a.akamaihd.net.edgesuite.net riot;
pls.patch.station.sony.com sony;
gs2.ww.prod.dl.playstation.net sony;
*.content.steampowered.com steam;
content1.steampowered.com steam;
content2.steampowered.com steam;
content3.steampowered.com steam;
content4.steampowered.com steam;
content5.steampowered.com steam;
content6.steampowered.com steam;
content7.steampowered.com steam;
content8.steampowered.com steam;
cs.steampowered.com steam;
steamcontent.com steam;
client-download.steampowered.com steam;
*.hsar.steampowered.com.edgesuite.net steam;
*.akamai.steamstatic.com steam;
content-origin.steampowered.com steam;
clientconfig.akamai.steamtransparent.com steam;
steampipe.akamaized.net steam;
edgecast.steamstatic.com steam;
steam.apac.qtlglb.com.mwcloudcdn.com steam;
*.cs.steampowered.com steam;
*.edgecast.steamstatic.com steam;
*.steamcontent.com steam;
cdn1-sea1.valve.net steam;
cdn2-sea1.valve.net steam;
*.steam-content-dnld-1.apac-1-cdn.cqloud.com steam;
steam.apac.qtlglb.com steam;
edge.steam-dns.top.comcast.net steam;
edge.steam-dns-2.top.comcast.net steam;
steamcdn-a.akamaihd.net steam;
steam.naeu.qtlglb.com steam;
steampipe-kr.akamaized.net steam;
steam.ix.asn.au steam;
steam.eca.qtlglb.com steam;
steam.cdn.on.net steam;
*.cdn.ubi.com uplay;
d3rmjivj4k4f0t.cloudfront.net twitch;
addons.forgesvc.net twitch;
media.forgecdn.net twitch;
files.forgecdn.net twitch;
dl1.wargaming.net wargaming;
dl2.wargaming.net wargaming;
wg.gcdn.co wargaming;
wgusst-na.wargaming.net wargaming;
wgusst-eu.wargaming.net wargaming;
update-v4r4h10x.worldofwarships.com wargaming;
dl-wows-ak.wargaming.net wargaming;
wgus-wotasia.wargaming.net wargaming;
dl-wot-ak.wargaming.net wargaming;
dl-wot-gc.wargaming.net wargaming;
dl-wot-cdx.wargaming.net wargaming;
officecdn.microsoft.com wsus;
*.windowsupdate.com wsus;
windowsupdate.com wsus;
*.dl.delivery.mp.microsoft.com wsus;
dl.delivery.mp.microsoft.com wsus;
*.update.microsoft.com wsus;
*.do.dsp.mp.microsoft.com wsus;
*.microsoft.com.edgesuite.net wsus;
assets1.xboxlive.com xboxlive;
assets2.xboxlive.com xboxlive;
dlassets.xboxlive.com xboxlive;
xboxone.loris.llnwd.net xboxlive;
*.xboxone.loris.llnwd.net xboxlive;
xboxone.vo.llnwd.net xboxlive;
xbox-mbr.xboxlive.com xboxlive;
assets1.xboxlive.com.nsatc.net xboxlive;
}
Executing hook /hooks/entrypoint-pre.d/19_doc_root_setup
/var/www/html already exists.
Setting document root to /var/www/html
Executing hook /hooks/entrypoint-pre.d/20_perms_check.sh
Checking permissions (This may take a long time if the permissions are incorrect on large caches)...
Data Information Script Thing
Hello for those interested I wrote a quick script here: https://github.com/DarrenRainey/Script-Bin/blob/master/Sys%20Admin/Steamcache-Monolithic-Data-Stats.sh to display lancache stats as seen below:
Hit ratio inflated due to slice
Hello friends!
Using nginx's slice module means that:
the first slice is fetched in the main request, while all other slices are fetched in subrequests. Since you log only the main request (by default), the variable $upstream_cache_status returns cache status of the first slice, which is likely to be cached.
arut - https://trac.nginx.org/nginx/ticket/1200
This paints a rosy picture when logging via lancachenet/logstash and visualising in Kibana, showing hit ratios that are much better than they really are.
Simply enabling logging of subrequests with log_subrequest on would mean counting request bytes twice, once for the main request for e.g. 5MB, and then again for the 5x 1MB subrequests, throwing the graphs off another way.
Nginx doesn't have an equivalent to Apache's IS_SUBREQ variable, so some other way of differentiating main request log lines from subrequest lines is needed, potentially adding a header to one or other request and logging that, as discussed with @GotenXiao at UK LAN Techs hack weekend 2.0.
Once we can differentiate between log lines, in Kibana the graphs will need to be updated so that:
- the "hit / miss" pie is drawn from subrequest log lines only
- the "total bytes served" metric is drawn from main request log lines only
- the "internet bandwidth saved" metric is drawn from subrequest log lines marked as a hit only
... and many of the other graphs too, to avoid counting bytes twice as described above.
Thanks!
watchlog.sh in monolithic?
Describe the issue you are having
Hi guys,
I must admit I'm new to linux, docker as well as steamcache but I successfully managed to deploy a server based on monolithic for a small LAN party (10 friends) over the last weekend, and it worked well enough aside from the slow initial speeds others have reported.
I've been wondering if there is a better way to visualise cache hits vs misses, as opposed to trying to follow the tail output of access.log. I noticed that the steamcache container has a script that colours hits/misses called watchlog.sh. My attempts at modifying this for monolithic's access.log didn't work too well at all.
It's probably pretty far down the list of issues and features you're planning on at the moment, but is there anything you can suggest or a similar script for this container?
Thanks!
LanCache.net shows incorrect docker container names
Describe the issue you are having
https://lancache.net shows docker commands which use a nonexistent repository name (lancache/<container> rather than lancachenet/<container>).
This results in a "repository does not exist" error and probably some confused users :)
Question
Can we use this to cache other streaming website like youtube or social media like facebook?
Steam client won't update
Describe the issue you are having
On one machine the client says it is up to date but hasn't updated since I started using the cache. I just did a fresh install on another machine and it said I needed to be online to update the client. As soon as I point the DNS away from the cache it updated fine.
How are you running the container(s)?
docker run --restart unless-stopped --name lancache --detach -v /cache/data:/data/cache -v /cache/logs:/data/logs -p 192.168.1.81:80:80 -p 192.168.1.82:80:80 -p 192.168.1.83:80:80 -p 192.168.1.84:80:80 -e CACHE_MEM_SIZE=4000m -e CACHE_DISK_SIZE=1000g steamcache/monolithic:latest
docker run --restart unless-stopped --name wsuscache --detach -v /wsus/data:/data/cache -v /wsus/logs:/data/logs -p 192.168.1.85:80:80 -e CACHE_MEM_SIZE=4000m -e CACHE_DISK_SIZE=500g -e CACHE_MAX_AGE=183d steamcache/monolithic:latest
docker run --restart unless-stopped --name sniproxy --detach -p 443:443 steamcache/sniproxy:latest
DNS Configuration
docker run --restart unless-stopped --name steamcache-dns --detach -p 192.168.1.80:53:53/udp -e USE_GENERIC_CACHE=true -e LANCACHE_IP="192.168.1.81 192.168.1.82 192.168.1.83 192.168.1.84" -e WSUSCACHE_IP="192.168.1.85" -e UPSTREAM_DNS=1.1.1.1 steamcache/steamcache-dns:latest
Error upon running lancachenet/monolithic
I have just followed the first step described on lancache.net docker run --restart unless-stopped --name lancache -v /cache/data:/data/cache -v /cache/logs:/data/logs -p 80:80 lancachenet/monolithic:latest and I am getting the following error messages at the bottom:
Checking nginx config
nginx: [emerg] could not build map_hash, you should increase map_hash_bucket_size: 32
nginx: configuration file /etc/nginx/nginx.conf test failed
ERROR: hook /hooks/supervisord-pre.d/99_config_check.sh} returned a non-zero exit status '0'
Am I doing something wrong?
Seemingly slow download speeds during cache warmup... thoughts?
Describe the issue you are having
Was testing this out for an upcoming LAN party I help run tech support on, and everything is working pretty well, but I noticed on my router that I was only average 115 Mbit/s vs normally getting 200 mbps (my max bandwidth) when downloading from Steam. Obviously this could just be a fluke or some issue somewhere on the internet, but I wanted to know if there's anything simple I can do from my server to potentially troubleshoot it? My first thought was to load steamcmd just to make sure there wasn't anything weird going on on my network (really shouldn't be as router, server, and gaming pc are all on the same switch)
How are you running the container(s)?
docker run \
--restart unless-stopped \
--name cache-steam \
-v /data/cache:/data/cache \
-v /data/logs:/data/logs \
-e CACHE_MEM_SIZE=8000m \
-e CACHE_DISK_SIZE=1500g \
-p 80:80 \
steamcache/monolithic:latest
DNS Configuration
docker run --restart unless-stopped --name steamcache-dns -p 53:53/udp -e USE_GENERIC_CACHE=true -e LANCACHE_IP=192.168.100.223 steamcache/steamcache-dns:latest
[RFE] Add FAQ entry for lancache-autofill?
Feature Request/Question
I would like to try and manually pre-seed the cache before an event, due to the fact the event will have terrible connectivity, and I have 1Gbps/1Gbps connection at home.
I know I could just uninstall/reinstall all of my games, but I was hoping to have some way to run a command or poke a URL/API to ask the cache to download the content, so will act more like an initial download server during the event as well as a cache.
I would of course prefer to turn up to the event with a cache that already has a large amount of the content in the cache, so we can reduce the initial load on the events crappy internet connection.
Run 2nd cache for WSUS
Describe the issue you are having
I would like to run a 2nd cache just for WSUS updates so that I can have them stored on slower storage with a shorter retention period but I am not sure of the correct context for the commands, do I just run a 2nd container with default settings and control access to it via the DNS?
How are you running the container(s)?
docker run --restart unless-stopped --name lancache --detach -v /cache/data:/data/cache -v /cache/logs:/data/logs -p 192.168.0.81:80:80 -p 192.168.0.82:80:80 -p 192.168.0.83:80:80 -p 192.168.0.84:80:80 -e CACHE_MEM_SIZE=4000m -e CACHE_DISK_SIZE=1000g steamcache/monolithic:latest
docker run --restart unless-stopped --name wsuscache --detach -v /wsus/data:/data/cache -v /wsus/logs:/data/logs -p 192.168.0.85:80:80 -e CACHE_MEM_SIZE=4000m -e CACHE_DISK_SIZE=500g -e CACHE_MAX_AGE 183d steamcache/monolithic:latest
DNS Configuration
docker run --restart unless-stopped --name steamcache-dns --detach -p 192.168.0.80:53:53/udp -e USE_GENERIC_CACHE=true -e LANCACHE_IP="192.168.0.81 192.168.0.82 192.168.0.83 192.168.0.84" -e WSUSCACHE_IP="192.168.0.85" -e UPSTREAM_DNS=1.1.1.1 steamcache/steamcache-dns:latest
docker run --restart unless-stopped --name sniproxy --detach -p 443:443 steamcache/sniproxy:latest
Output of container(s)
$ docker run --restart unless-stopped --name wsuscache --detach -v /wsus/data:/data/cache -v /wsus/logs:/data/logs -p 192.168.0.85:80:80 -e CACHE_MEM_SIZE=4000m -e CACHE_DISK_SIZE=500g -e CACHE_MAX_AGE 183d steamcache/monolithic:latest
Unable to find image '183d:latest' locally
docker: Error response from daemon: pull access denied for 183d, repository does not exist or may require 'docker login'.
See 'docker run --help'
cache access (miss content)
Describe the issue you are having
I just getting miss on the log, can anyone help?
How are you running the container(s)?
docker run --restart unless-stopped --name steamcache-dns --detach -p 53:53/udp -e USE_GENERIC_CACHE=true -e LANCACHE_IP=10.10.10.2 steamcache/steamcache-dns:latest
docker run --restart unless-stopped --name lancache --detach -v /cache/data:/data/cache -v /cache/logs:/data/logs -p 80:80 steamcache/monolithic:latest
DNS Configuration
docker run --name steamcache-dns -p 10.10.10.3:53:53/udp -e USE_GENERIC_CACHE=true -e LANCACHE_IP=10.10.10.2 steamcache/steamcache-dns:latest
Output of container(s)
[steam] 10.10.10.254 / - - - [20/Apr/2019:03:37:00 -0300] "GET /depot/1/chunk/e1156c47385927ff161f4d9033dd763c79b01bc0?l=9&e=1555976254&sid=26588943&h=16d431d51cb630ebbdd818cd5e19874bd71749c0 HTTP/1.1" 200 854912 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-gru1.steamcontent.com" "-"
[steam] 10.10.10.254 / - - - [20/Apr/2019:03:37:01 -0300] "GET /depot/1/chunk/09171ba34a65c57f4de83e1604ddcff7f172805d?l=9&e=1555976254&sid=26588943&h=16d431d51cb630ebbdd818cd5e19874bd71749c0 HTTP/1.1" 200 1039968 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-gru1.steamcontent.com" "-"
[steam] 10.10.10.254 / - - - [20/Apr/2019:03:37:01 -0300] "GET /depot/1/chunk/fa1e15f5436299127de5a8f9aa4829b88d8caddd?l=9&e=1555976254&sid=26588943&h=16d431d51cb630ebbdd818cd5e19874bd71749c0 HTTP/1.1" 200 517584 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-gru1.steamcontent.com" "-"
[steam] 10.10.10.254 / - - - [20/Apr/2019:03:37:01 -0300] "GET /depot/1/chunk/c955fff3ff0771b457d4ca92f132b3647dc7baf8?l=9&e=1555976254&sid=26588943&h=16d431d51cb630ebbdd818cd5e19874bd71749c0 HTTP/1.1" 200 7024 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-gru1.steamcontent.com" "-"
[steam] 10.10.10.254 / - - - [20/Apr/2019:03:37:01 -0300] "GET /depot/1/chunk/9731ca2fdcfe85b2ac497654e8d47d8612664f1c?l=9&e=1555976254&sid=26588943&h=16d431d51cb630ebbdd818cd5e19874bd71749c0 HTTP/1.1" 200 5776 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-gru1.steamcontent.com" "-"
[steam] 10.10.10.254 / - - - [20/Apr/2019:03:37:01 -0300] "GET /depot/1/chunk/406153bf77af6ffe5f242e636ffe25a88546f38d?l=9&e=1555976254&sid=26588943&h=16d431d51cb630ebbdd818cd5e19874bd71749c0 HTTP/1.1" 200 2400 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-gru1.steamcontent.com" "-"
[steam] 10.10.10.254 / - - - [20/Apr/2019:03:37:01 -0300] "GET /depot/1/chunk/feb833e7a609f5f2af9e3cb16563d557a9b81f8a?l=9&e=1555976254&sid=26588943&h=16d431d51cb630ebbdd818cd5e19874bd71749c0 HTTP/1.1" 200 2272 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-gru1.steamcontent.com" "-"
[steam] 10.10.10.254 / - - - [20/Apr/2019:03:37:01 -0300] "GET /depot/1/chunk/05d7240cc7f5be6d1af77945aa5cd045778a80c2?l=9&e=1555976254&sid=26588943&h=16d431d51cb630ebbdd818cd5e19874bd71749c0 HTTP/1.1" 200 5472 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-gru1.steamcontent.com" "-"
[steam] 10.10.10.254 / - - - [20/Apr/2019:03:37:01 -0300] "GET /depot/1/chunk/2584202e0cc93e0269bc0cb185630cee07718b75?l=9&e=1555976254&sid=26588943&h=16d431d51cb630ebbdd818cd5e19874bd71749c0 HTTP/1.1" 200 2448 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-gru1.steamcontent.com" "-"
[steam] 10.10.10.254 / - - - [20/Apr/2019:03:37:02 -0300] "GET /depot/1/chunk/0f291eb76fffaf8605ce13b9cec2ab9af8c26efa?l=9&e=1555976254&sid=26588943&h=16d431d51cb630ebbdd818cd5e19874bd71749c0 HTTP/1.1" 200 304 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-gru1.steamcontent.com" "-"
[steam] 10.10.10.254 / - - - [20/Apr/2019:03:37:02 -0300] "GET /depot/1/chunk/5a34f9789af44c921b63223cf26be26aa353d984?l=9&e=1555976254&sid=26588943&h=16d431d51cb630ebbdd818cd5e19874bd71749c0 HTTP/1.1" 200 304 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-gru1.steamcontent.com" "-"
[steam] 10.10.10.254 / - - - [20/Apr/2019:03:37:02 -0300] "GET /depot/1/chunk/fadd5742d70210f87905e72960d16d5ed27cb1e3?l=9&e=1555976254&sid=26588943&h=16d431d51cb630ebbdd818cd5e19874bd71749c0 HTTP/1.1" 200 49904 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-gru1.steamcontent.com" "-"
[steam] 10.10.10.254 / - - - [20/Apr/2019:03:37:02 -0300] "GET /depot/1/chunk/87eb2cefc597790161725066ae2dedec4ab13f4b?l=9&e=1555976254&sid=26588943&h=16d431d51cb630ebbdd818cd5e19874bd71749c0 HTTP/1.1" 200 414816 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-gru1.steamcontent.com" "-"
[steam] 10.10.10.254 / - - - [20/Apr/2019:03:37:02 -0300] "GET /depot/1/chunk/477602c4c50e2bc882b9241a2bec7d13a7c6e277?l=9&e=1555976254&sid=26588943&h=16d431d51cb630ebbdd818cd5e19874bd71749c0 HTTP/1.1" 200 10000 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-gru1.steamcontent.com" "-"
[steam] 10.10.10.254 / - - - [20/Apr/2019:03:37:02 -0300] "GET /depot/1/chunk/2069d7432766cfbd8965d44919a1aacf0d4b5b9c?l=9&e=1555976254&sid=26588943&h=16d431d51cb630ebbdd818cd5e19874bd71749c0 HTTP/1.1" 200 14560 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-gru1.steamcontent.com" "-"
[steam] 10.10.10.254 / - - - [20/Apr/2019:03:37:02 -0300] "GET /depot/1/chunk/b55333e8710847cd489e625a92fde59d02a6b834?l=9&e=1555976254&sid=26588943&h=16d431d51cb630ebbdd818cd5e19874bd71749c0 HTTP/1.1" 200 28896 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-gru1.steamcontent.com" "-"
[steam] 10.10.10.254 / - - - [20/Apr/2019:03:37:02 -0300] "GET /depot/1/chunk/a32a3f6cb3575397792e1dfcb50db0d5c55502f1?l=9&e=1555976254&sid=26588943&h=16d431d51cb630ebbdd818cd5e19874bd71749c0 HTTP/1.1" 200 826800 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-gru1.steamcontent.com" "-"
Looking to switch to this, but need some clarification.
Currently I am using bntjah/lancache, which is good, but it's having a lot of issues with steam connections being limited to 2/3 MB/s because of the connection pool, and no offense to them, but they haven't been very helpful in helping me diagnose it.
I see here that you guys support other services such as blizzard, origin, etc. My main question, does this all run through one IP? I did see something about multiple IPs being support to solve a connection pool issue, but it's not very clear.
Bntjah requires 18 IPs or so. One for the lancache IP, then virtual IPs that it hands off requests to to support more throughput. This all works great except for when more than 2 people try to download new games or updates from the internet within a period of time for steam, just steam.
And I guess my second question is, is there a comprehensive and straight forward guide to setting this up? I'm trying to find a solution for a lan center with over 100 PCs where we nearly cap out on the weekends and have a lot of traffic.
I'm usually okay with sitting down and learning all this stuff, and I've already spent hours trying to get different lancaches to work, but we're at a point where we have been open for a while yet we can't guarantee fast downloads because our lancache keeps dying every couple weeks. Besides that, my main job isn't this, but we're a small company and we need people wearing many hats.
Thanks for the help.
Windows Update cache not working
Describe the issue you are having
Using a very basic setup as proposed in the manual, my caches are working just fine. However Windows Update seems to be troubled. I get the error message "We couldn't connect to the update service. We'll try again later, or you can check now. If it still doesn't work, make sure you're connected to the Internet". It appears every time i click on the check for updates button.
I believe the problem has to do with the sniproxy. I can see that it's passing some traffic along, but my PC never tries to actually download anything:
2019-06-08 13:32:50 X.X.X.X:7257 -> Y.Y.Y.Y:443 -> 13.68.93.109:443 [sls.update.microsoft.com] 4027/4027 bytes tx 507/507 bytes rx 0.514 seconds
2019-06-08 13:32:50 X.X.X.X:7259 -> Y.Y.Y.Y:443 -> 13.74.179.117:443 [sls.update.microsoft.com] 4027/4027 bytes tx 507/507 bytes rx 0.289 seconds
2019-06-08 13:32:50 X.X.X.X:7260 -> Y.Y.Y.Y:443 -> 13.74.179.117:443 [sls.update.microsoft.com] 4027/4027 bytes tx 507/507 bytes rx 0.249 seconds
2019-06-08 13:32:50 X.X.X.X:7261 -> Y.Y.Y.Y:443 -> 13.74.179.117:443 [sls.update.microsoft.com] 4027/4027 bytes tx 507/507 bytes rx 0.236 seconds
2019-06-08 13:32:51 X.X.X.X:7262 -> Y.Y.Y.Y:443 -> 13.74.179.117:443 [sls.update.microsoft.com] 4027/4027 bytes tx 507/507 bytes rx 0.247 seconds
2019-06-08 13:32:51 X.X.X.X:7263 -> Y.Y.Y.Y:443 -> 13.68.93.109:443 [sls.update.microsoft.com] 4027/4027 bytes tx 507/507 bytes rx 0.483 seconds
2019-06-08 13:32:51 X.X.X.X:7264 -> Y.Y.Y.Y:443 -> 13.74.179.117:443 [sls.update.microsoft.com] 4027/4027 bytes tx 507/507 bytes rx 0.240 seconds
2019-06-08 13:32:52 X.X.X.X:7265 -> Y.Y.Y.Y:443 -> 13.74.179.117:443 [sls.update.microsoft.com] 4027/4027 bytes tx 507/507 bytes rx 0.238 seconds
DNS is working fine. My PC resolves the relevant hostnames to the IP address of the lancache/sniproxy container
As a temporary workaround, i started the DNS container with DISABLE_WSUS=true. However it would be great if the problem could be resolved somehow, because windows update caching would really speed up my network (3+ Windows 10 clients and a slow connection are no fun at all)
Maybe whitelisting sls.update.microsoft.com specifically could already be enough, because it looks like Windows first queries this address via HTTPS to check for available updates before downloading them (from another address) over HTTP. The HTTPS traffic can't be cached anyway.
How are you running the container(s)?
Pretty much exactly as described under "Simple Full Stack Setup". However i use seperate IPs to bind the DNS and HTTP/HTTPS ports to.
DNS Configuration
Statically configured my PC to resolve via the steamcache-dns container IP address. This is working fine:
C:\Users\xxxxxx>nslookup sls.update.microsoft.com
Server: 396d88284d7c
Address: Z.Z.Z.11
Non-authoritative answer:
Name: wsus.cache.steamcache.net
Address: Z.Z.Z.12
Aliases: sls.update.microsoft.com
Output of container(s)
see description above, no additional relevant output
Origin not cached, all else is
Describe the issue you are having
I am not seeing games on Origin picked up by the cache, but other services such as Steam and Blizzard's Battle.net app are cached as intended. This is in Stockholm, Sweden. Typical behavior for HTTPS traffic, but Origin shouldn't be downloading games over HTTPS?
How are you running the container(s)?
Exactly as per the readme except the location of the storage for the lancache container, since my host has it on /lancache and not /data.
DNS Configuration
The run command is directly from the readme. Device points at the host IP.
Output of container(s)
I'll happily dump any logs that you might need, just type out the path of them to me.
Does not download files to disk
Describe the issue you are having
Moved from generic to this, cant get to work, resolves IP's correctly but doesnt seem to save any data to disk I get the below error messages in error.log I see it creates the folders as the www-data user which doesnt exist on my docker host, except under uid 33 (default debian users?)
they are all chmod'd 777 however. it is using a NFS Share on a NAS
2018/12/20 03:24:13 [crit] 192#192: *35 open() "/data/cache/cache/99/ac/57f4a2d49f8869000a7ded344cc4ac99" failed (13: Permission denied), client: 10.0.0.114, server: , request: "GET /serverlist/77/20/ HTTP/1.1", host: "valve325.steamcontent.com"
2018/12/20 03:24:13 [crit] 193#193: *36 open() "/data/cache/cache/99/ac/57f4a2d49f8869000a7ded344cc4ac99" failed (13: Permission denied), client: 10.0.0.102, server: , request: "GET /serverlist/77/20/ HTTP/1.1", host: "valve2300.steamcontent.com"
How are you running the container(s)?
version: '2.4'
services:
sniproxy:
image: steamcache/sniproxy
container_name: sniproxy
restart: unless-stopped
environment:
- TZ=America/Chicago
ports:
- "10.0.0.82:443:443"
- "10.0.0.83:443:443"
mem_limit: 20M
cache-monolithic:
image: steamcache/monolithic
container_name: cache-monolithic
restart: unless-stopped
depends_on:
- sniproxy
volumes:
- /mnt/cache/data:/data/cache
- /mnt/cache/logs:/data/logs
environment:
- TZ=America/Chicago
- PGID=1000
- PUID=1000
ports:
- "10.0.0.82:80:80"
mem_limit: 1024M
dns:
- 1.1.1.1
- 9.9.9.9
DNS Configuration
Output of container(s)
root@docker:~# docker-compose up cache-monolithic
sniproxy is up-to-date
Starting cache-monolithic ... done
Attaching to cache-monolithic
cache-monolithic | Executing hook /hooks/entrypoint-pre.d/10_setup.sh
cache-monolithic | Executing hook /hooks/entrypoint-pre.d/15_generate_maps.sh
cache-monolithic | HEAD is now at 63a12a6 Merge pull request #47 from rotanid/wargaming-fixes
cache-monolithic | Reading cache apple from apple.txt
cache-monolithic | Reading cache arenanet from arenanet.txt
cache-monolithic | Reading cache blizzard from blizzard.txt
cache-monolithic | Reading cache daybreak from daybreak.txt
cache-monolithic | Reading cache frontier from frontier.txt
cache-monolithic | Reading cache hirez from hirez.txt
cache-monolithic | Reading cache nexusmods from nexusmods.txt
cache-monolithic | Reading cache nintendo from nintendo.txt
cache-monolithic | Reading cache origin from origin.txt
cache-monolithic | Reading cache renegadex from renegadex.txt
cache-monolithic | Reading cache riot from riot.txt
cache-monolithic | Reading cache rockstar from rockstar.txt
cache-monolithic | Reading cache sony from sony.txt
cache-monolithic | Reading cache steam from steam.txt
cache-monolithic | Reading cache uplay from uplay.txt
cache-monolithic | Reading cache twitch from twitchapp.txt
cache-monolithic | Reading cache wargaming from wargaming.net.txt
cache-monolithic | Reading cache wsus from windowsupdates.txt
cache-monolithic | Reading cache xboxlive from xboxlive.txt
cache-monolithic | map $http_host $cacheidentifier {
cache-monolithic | hostnames;
cache-monolithic | default $http_host;
cache-monolithic | assetcdn.101.arenanetworks.com arenanet;
cache-monolithic | assetcdn.102.arenanetworks.com arenanet;
cache-monolithic | assetcdn.103.arenanetworks.com arenanet;
cache-monolithic | dist.blizzard.com blizzard;
cache-monolithic | dist.blizzard.com.edgesuite.net blizzard;
cache-monolithic | llnw.blizzard.com blizzard;
cache-monolithic | edgecast.blizzard.com blizzard;
cache-monolithic | blizzard.vo.llnwd.net blizzard;
cache-monolithic | blzddist1-a.akamaihd.net blizzard;
cache-monolithic | blzddist2-a.akamaihd.net blizzard;
cache-monolithic | blzddist3-a.akamaihd.net blizzard;
cache-monolithic | blzddist4-a.akamaihd.net blizzard;
cache-monolithic | level3.blizzard.com blizzard;
cache-monolithic | nydus.battle.net blizzard;
cache-monolithic | edge.blizzard.top.comcast.net blizzard;
cache-monolithic | cdn.blizzard.com blizzard;
cache-monolithic | *.cdn.blizzard.com blizzard;
cache-monolithic | # Daybreak games daybreak;
cache-monolithic | # daybreak;
cache-monolithic | # Do NOT cache manifest.patch.daybreakgames.com daybreak;
cache-monolithic | # daybreak;
cache-monolithic | #PS2 daybreak;
cache-monolithic | pls.patch.daybreakgames.com daybreak;
cache-monolithic | ccs.cdn.wup.shop.nintendo.com nintendo;
cache-monolithic | pushmo.hac.lp1.eshop.nintendo.net nintendo;
cache-monolithic | ecs-lp1.hac.shop.nintendo.net nintendo;
cache-monolithic | receive-lp1.dg.srv.nintendo.net nintendo;
cache-monolithic | aqua.hac.lp1.d4c.nintendo.net nintendo;
cache-monolithic | atum.hac.lp1.d4c.nintendo.net nintendo;
cache-monolithic | bugyo.hac.lp1.eshop.nintendo.net nintendo;
cache-monolithic | tagaya.hac.lp1.eshop.nintendo.net nintendo;
cache-monolithic | # WARNING: Origin has been seen downloading https client downloads on origin-a.akamaihd.net. A solution should be in place to forward https to the origin server (eg sniproxy) origin;
cache-monolithic | origin-a.akamaihd.net origin;
cache-monolithic | akamai.cdn.ea.com origin;
cache-monolithic | lvlt.cdn.ea.com origin;
cache-monolithic | river.data.ea.com origin;
cache-monolithic | origin-a.akamaihd.net.edgesuite.net origin;
cache-monolithic | rxp-fl.cncirc.net renegadex;
cache-monolithic | rxp-chi.cncirc.net renegadex;
cache-monolithic | rxp-nz.cncirc.net renegadex;
cache-monolithic | rxp-bgr.cncirc.net renegadex;
cache-monolithic | rxp-fr.cncirc.net renegadex;
cache-monolithic | rxp-nyc.cncirc.net renegadex;
cache-monolithic | rxp-uk.cncirc.net renegadex;
cache-monolithic | rxp-sg.cncirc.net renegadex;
cache-monolithic | rxp-la.cncirc.net renegadex;
cache-monolithic | rxp-fin.cncirc.net renegadex;
cache-monolithic | denver1.renegade-x.com renegadex;
cache-monolithic | l3cdn.riotgames.com riot;
cache-monolithic | worldwide.l3cdn.riotgames.com riot;
cache-monolithic | riotgamespatcher-a.akamaihd.net riot;
cache-monolithic | riotgamespatcher-a.akamaihd.net.edgesuite.net riot;
cache-monolithic | pls.patch.station.sony.com sony;
cache-monolithic | gs2.ww.prod.dl.playstation.net sony;
cache-monolithic | *.content.steampowered.com steam;
cache-monolithic | content1.steampowered.com steam;
cache-monolithic | content2.steampowered.com steam;
cache-monolithic | content3.steampowered.com steam;
cache-monolithic | content4.steampowered.com steam;
cache-monolithic | content5.steampowered.com steam;
cache-monolithic | content6.steampowered.com steam;
cache-monolithic | content7.steampowered.com steam;
cache-monolithic | content8.steampowered.com steam;
cache-monolithic | cs.steampowered.com steam;
cache-monolithic | steamcontent.com steam;
cache-monolithic | client-download.steampowered.com steam;
cache-monolithic | *.hsar.steampowered.com.edgesuite.net steam;
cache-monolithic | *.akamai.steamstatic.com steam;
cache-monolithic | content-origin.steampowered.com steam;
cache-monolithic | clientconfig.akamai.steamtransparent.com steam;
cache-monolithic | steampipe.akamaized.net steam;
cache-monolithic | edgecast.steamstatic.com steam;
cache-monolithic | steam.apac.qtlglb.com.mwcloudcdn.com steam;
cache-monolithic | *.cs.steampowered.com steam;
cache-monolithic | *.edgecast.steamstatic.com steam;
cache-monolithic | *.steamcontent.com steam;
cache-monolithic | cdn1-sea1.valve.net steam;
cache-monolithic | cdn2-sea1.valve.net steam;
cache-monolithic | *.steam-content-dnld-1.apac-1-cdn.cqloud.com steam;
cache-monolithic | steam.apac.qtlglb.com steam;
cache-monolithic | edge.steam-dns.top.comcast.net steam;
cache-monolithic | edge.steam-dns-2.top.comcast.net steam;
cache-monolithic | steamcdn-a.akamaihd.net steam;
cache-monolithic | steam.naeu.qtlglb.com steam;
cache-monolithic | steampipe-kr.akamaized.net steam;
cache-monolithic | steam.ix.asn.au steam;
cache-monolithic | steam.eca.qtlglb.com steam;
cache-monolithic | *.cdn.ubi.com uplay;
cache-monolithic | d3rmjivj4k4f0t.cloudfront.net twitch;
cache-monolithic | addons.forgesvc.net twitch;
cache-monolithic | media.forgecdn.net twitch;
cache-monolithic | files.forgecdn.net twitch;
cache-monolithic | dl1.wargaming.net wargaming;
cache-monolithic | dl2.wargaming.net wargaming;
cache-monolithic | wg.gcdn.co wargaming;
cache-monolithic | wgusst-na.wargaming.net wargaming;
cache-monolithic | wgusst-eu.wargaming.net wargaming;
cache-monolithic | update-v4r4h10x.worldofwarships.com wargaming;
cache-monolithic | dl-wows-ak.wargaming.net wargaming;
cache-monolithic | wgus-wotasia.wargaming.net wargaming;
cache-monolithic | officecdn.microsoft.com wsus;
cache-monolithic | *.windowsupdate.com wsus;
cache-monolithic | windowsupdate.com wsus;
cache-monolithic | *.dl.delivery.mp.microsoft.com wsus;
cache-monolithic | dl.delivery.mp.microsoft.com wsus;
cache-monolithic | *.update.microsoft.com wsus;
cache-monolithic | *.do.dsp.mp.microsoft.com wsus;
cache-monolithic | *.microsoft.com.edgesuite.net wsus;
cache-monolithic | assets1.xboxlive.com xboxlive;
cache-monolithic | assets2.xboxlive.com xboxlive;
cache-monolithic | dlassets.xboxlive.com xboxlive;
cache-monolithic | xboxone.loris.llnwd.net xboxlive;
cache-monolithic | *.xboxone.loris.llnwd.net xboxlive;
cache-monolithic | xboxone.vo.llnwd.net xboxlive;
cache-monolithic | images-eds.xboxlive.com xboxlive;
cache-monolithic | xbox-mbr.xboxlive.com xboxlive;
cache-monolithic | }
cache-monolithic | Executing hook /hooks/entrypoint-pre.d/19_doc_root_setup
cache-monolithic | /var/www/html already exists.
cache-monolithic | Setting document root to /var/www/html
cache-monolithic | Executing hook /hooks/entrypoint-pre.d/20_perms_check.sh
cache-monolithic | Checking permissions (This may take a long time if the permissions are incorrect on large caches)...
cache-monolithic | Permisions ok
cache-monolithic | Executing hook /hooks/entrypoint-pre.d/20_ssl_setup
cache-monolithic | Not enabling SSL as neither key nor cert provided.
cache-monolithic | Executing hook /hooks/supervisord-pre.d/20_test_files_setup
cache-monolithic | Checking if /var/www/html is empty - Directory not empty.. don't touch content
cache-monolithic | Executing hook /hooks/supervisord-pre.d/21_cleanup_log_files
cache-monolithic | Cleaning up log files older than 3560 days
cache-monolithic | Executing hook /hooks/supervisord-pre.d/99_config_check.sh
cache-monolithic | Checking nginx config
cache-monolithic | nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
cache-monolithic | nginx: configuration file /etc/nginx/nginx.conf test is successful
cache-monolithic | Config check successful
cache-monolithic | 2018-12-20 03:24:11,149 CRIT Set uid to user 0
IPv6
We're had problems setting this hole thing up, till we disabled the ipv6 in the windows clients for dns.
We suggest an FAQ or a hint on the install instructions that people should disable their ipv6 on clients?!
Thanks for lancache!
Unable to resolve github.com
Describe the issue you are having
container is unable to resolve github.com so never downloads lancache domians file - therefore all requests for content also get cannot be resolved
I have tried removing the container, redownloading and clearing the cache folder etc already.
From the host machine I can ping github.com so I am unsure why the container cannot get through to this.
Worth noting this worked up until roughly a week ago and then suddenly stopped so steam downloads etc tend to just sit at zero till they fail. All other containers are having no issues connecting to the web etc so any help is greatly appreciated. Thanks
How are you running the container(s)?
docker run -d --name='Monolithic-Cache' --net='br0' --ip='192.168.11.202' --privileged=true -e TZ="Europe/London" -e HOST_OS="Unraid" -e 'UPSTREAM_DNS'='1.1.1.1 8.8.8.8' -e 'CACHE_DISK_SIZE'='100g' -e 'BEAT_TIME'='1h' -v '/mnt/user/steamcache/':'/data/cache':'rw' -v '/mnt/user/appdata/steamcache/':'/data/logs':'rw' --restart unless-stopped --cap-add=NET_ADMIN 'steamcache/monolithic'
DNS Configuration
docker run -d --name='Monolithic-DNS' --net='br0' --ip='192.168.11.201' --privileged=true -e TZ="Europe/London" -e HOST_OS="Unraid" -e 'USE_GENERIC_CACHE'='true' -e 'UPSTREAM_DNS'='1.1.1.1' -e 'LANCACHE_IP'='192.168.11.202' -e 'STEAMCACHE_IP'='192.168.11.202 192.168.11.203 192.168.11.204 192.168.11.205 192.168.11.206 192.168.11.207 192.168.11.208 192.168.11.209 192.168.11.210 192.168.11.211' --restart unless-stopped 'steamcache/steamcache-dns'
Output of container(s)
Output from app log in unraid:
Executing hook /hooks/entrypoint-pre.d/10_setup.sh
Executing hook /hooks/entrypoint-pre.d/15_generate_maps.sh
Cloning into '.'...
fatal: unable to access 'https://github.com/uklans/cache-domains.git/': Could not resolve host: github.com
fatal: not a git repository (or any parent up to mount point /data)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /data)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
jq: error: Could not open file cache_domains.json: No such file or directory
map $http_host $cacheidentifier {
hostnames;
default $http_host;
}
Executing hook /hooks/entrypoint-pre.d/19_doc_root_setup
/var/www/html already exists.
Setting document root to /var/www/html
Executing hook /hooks/entrypoint-pre.d/20_perms_check.sh
Checking permissions (This may take a long time if the permissions are incorrect on large caches)...
Permisions ok
Executing hook /hooks/entrypoint-pre.d/20_ssl_setup
Not enabling SSL as neither key nor cert provided.
Executing hook /hooks/supervisord-pre.d/20_test_files_setup
Checking if /var/www/html is empty - Directory not empty.. don't touch content
Executing hook /hooks/supervisord-pre.d/21_cleanup_log_files
Cleaning up log files older than 3560 days
Executing hook /hooks/supervisord-pre.d/99_config_check.sh
Checking nginx config
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
Config check successful
2019-03-09 13:32:18,714 CRIT Set uid to user 0
fatal: unable to access 'https://github.com/uklans/cache-domains.git/': Could not resolve host: github.com
fatal: not a git repository (or any parent up to mount point /data)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /data)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
jq: error: Could not open file cache_domains.json: No such file or directory
Also, from access log:
2019/03/09 12:59:55 [error] 59#59: *30 level3.blizzard.com could not be resolved (110: Operation timed out), client: 192.168.11.3, server: , request: "HEAD /tpr/ovw/config/ff/c6/ffc61aeb5e49e0fba7f90245ca9d273b HTTP/1.1", host: "level3.blizzard.com"
Slow download speeds during cache warmup
Describe the issue you are having
My Steam Download Speeds through the cache are stuck at around 5-6 MBps, whilst without the cache I would be hitting around 10-11 MBps. I have tried adding a decent amount of IPs, as pointed out in the README, but that hasn't changed anything at all.
I am wondering whether I have done something wrong in the configuration of the docker containers or if there is an issue with steamcache.
The Steamcache is running in a KVM Container with 16 GB of RAM with 4 Broadwell Cores off a RAID0 ZFS Array dedicated to it, not hitting any IO Limitations. When pulling already cached data from the Steam Cache, I am getting around 80 MBps throughput with 2 Clients.
How are you running the container(s)?
HOST_IP=`hostname -I | cut -d' ' -f1`
sudo docker run --restart unless-stopped --name steamcache-dns --detach -p $HOST_IP:53:53/udp -e USE_GENERIC_CACHE=true -e LANCACHE_IP=$HOST_IP -e STEAMCACHE_IP="192.168.30.201 192.168.30.202 192.168.30.203 192.168.30.204 192.168.30.205 192.168.30.206 192.168.30.207 192.168.30.208 192.168.30.209 192.168.30.210 192.168.30.211 192.168.30.212 192.168.30.213 192.168.30.214 192.168.30.215 192.168.30.216 192.168.30.217 192.168.30.218 192.168.30.219 192.168.30.220" steamcache/steamcache-dns:latest
sudo docker run --restart unless-stopped --name lancache --detach -v /cache/data:/data/cache -v /cache/logs:/data/logs -p 80:80 -e CACHE_MEM_SIZE=16000m -e CACHE_DISK_SIZE=800g steamcache/monolithic:latest
sudo docker run --restart unless-stopped --name sniproxy --detach -p 443:443 steamcache/sniproxy:latest
echo Please configure your dhcp server to serve dns as $HOST_IP
DNS Configuration
Configured pfsense to point to steamcache-dns
IP Configuration
root@caching-server:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 1a:f5:5d:8d:81:e6 brd ff:ff:ff:ff:ff:ff
inet 192.168.30.201/24 brd 192.168.30.255 scope global ens18
valid_lft forever preferred_lft forever
inet 192.168.30.202/24 brd 192.168.30.255 scope global secondary ens18
valid_lft forever preferred_lft forever
inet 192.168.30.203/24 brd 192.168.30.255 scope global secondary ens18
valid_lft forever preferred_lft forever
inet 192.168.30.204/24 brd 192.168.30.255 scope global secondary ens18
valid_lft forever preferred_lft forever
inet 192.168.30.205/24 brd 192.168.30.255 scope global secondary ens18
valid_lft forever preferred_lft forever
inet 192.168.30.206/24 brd 192.168.30.255 scope global secondary ens18
valid_lft forever preferred_lft forever
inet 192.168.30.207/24 brd 192.168.30.255 scope global secondary ens18
valid_lft forever preferred_lft forever
inet 192.168.30.208/24 brd 192.168.30.255 scope global secondary ens18
valid_lft forever preferred_lft forever
inet 192.168.30.209/24 brd 192.168.30.255 scope global secondary ens18
valid_lft forever preferred_lft forever
inet 192.168.30.210/24 brd 192.168.30.255 scope global secondary ens18
valid_lft forever preferred_lft forever
inet 192.168.30.211/24 brd 192.168.30.255 scope global secondary ens18
valid_lft forever preferred_lft forever
inet 192.168.30.212/24 brd 192.168.30.255 scope global secondary ens18
valid_lft forever preferred_lft forever
inet 192.168.30.213/24 brd 192.168.30.255 scope global secondary ens18
valid_lft forever preferred_lft forever
inet 192.168.30.214/24 brd 192.168.30.255 scope global secondary ens18
valid_lft forever preferred_lft forever
inet 192.168.30.215/24 brd 192.168.30.255 scope global secondary ens18
valid_lft forever preferred_lft forever
inet 192.168.30.216/24 brd 192.168.30.255 scope global secondary ens18
valid_lft forever preferred_lft forever
inet 192.168.30.217/24 brd 192.168.30.255 scope global secondary ens18
valid_lft forever preferred_lft forever
inet 192.168.30.218/24 brd 192.168.30.255 scope global secondary ens18
valid_lft forever preferred_lft forever
inet 192.168.30.219/24 brd 192.168.30.255 scope global secondary ens18
valid_lft forever preferred_lft forever
inet 192.168.30.220/24 brd 192.168.30.255 scope global secondary ens18
valid_lft forever preferred_lft forever
inet6 2001:16b8:55c1:2900:18f5:5dff:fe8d:81e6/64 scope global dynamic mngtmpaddr noprefixroute
valid_lft 86385sec preferred_lft 14385sec
inet6 fe80::18f5:5dff:fe8d:81e6/64 scope link
valid_lft forever preferred_lft forever
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:40:55:ba:29 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
inet6 fe80::42:40ff:fe55:ba29/64 scope link
valid_lft forever preferred_lft forever
5: vethf943db7@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default
link/ether 52:5e:9e:05:60:64 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet6 fe80::505e:9eff:fe05:6064/64 scope link
valid_lft forever preferred_lft forever
7: veth568c974@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default
link/ether ee:3f:b4:37:ba:ab brd ff:ff:ff:ff:ff:ff link-netnsid 1
inet6 fe80::ec3f:b4ff:fe37:baab/64 scope link
valid_lft forever preferred_lft forever
9: vethdd6acbd@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default
link/ether 9e:06:e9:a3:f0:de brd ff:ff:ff:ff:ff:ff link-netnsid 2
inet6 fe80::9c06:e9ff:fea3:f0de/64 scope link
valid_lft forever preferred_lft forever
Documentation Error at DockerHub (Possibly Elsewhere)
Describe the issue you are having
At DockerHub in the "Simple Full Stack startup" section near the top of your instructions, you have a script to set up a typical full stack cache. However, this script fails because the name of one component (steamcache-DNS / lancache-dns) has changed.
I suspect this will work fine if the name is updated... Going to try now, hoping other config is not necessary.
How are you running the container(s)?
Attempted via said script.
DNS Configuration
Output of container(s)
docker: Error response from daemon: pull access denied for lancachenet/steamcache-dns, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.