skx / tweaked.io

The code behind http://tweaked.io/

tweaked.io

This repository contains the source of the (new) website http://tweaked.io/

Content Policy

The intention behind this site is that there will be one concise page of content for each of the major servers in common use.

For example:

• Apache 2.x.
• nginx.
• lighttpd.
• mysql.
• nfs.
• glusterfs.

I explicitly do not intend to cover the optimization of applications such as Magento, Wordpress, or similar. The reason for this is that too many variables are at play, and new releases would quickly render documentation obsolete.

By contrast most of the "well known" servers don't release too frequently.

(Although there certainly are some that do.)

Implementation Details

The website is built via the templer static-site generator.

If you have templer installed just run:

make serve

Then open your browser at http://localhost:4433/

Dependencies include:

• python-slimmer
  • Used to minify the CSS
• The following Perl-modules are templer dependencies:
  • Text::Markdown
  • HTML::Template

NOTE: If the python slimmer tool is unavailable the CSS will just be non-slimmed. This is a non-fatal error.

Comments

As the site is 100% static there is nothing dynamic at all, except for the display of inline comments at the foot of pages, which are inserted via javascript.

The comment-handling facility is divided into two parts:

• A simple server which receives new comments, and serves existing ones.
• A client-side javascript library which handles retrieving/posting new ones, via interactions with that remote server.

The commenting system was written for this site, but has been tidied up and made available in a standalone fashion now:

• https://github.com/skx/e-comments/

Contact

If wish to report issues please do feel free to mail me direct, or report them in the github tracker.

Licensing

All content on this site written by myself is assumed to be written under terms of the GNU General public license, version 2.

It is assumes contributions from others will be licensed under the same terms.

Steve

tweaked.io's Issues

Needs cover of glusterfs.

Gluster is slow.

Gluster can be made less slow, but usually at the cost of buffering tweaks, which increase the chance of issues.

Potential issues are that:

• Version changes, making it hard to assess things.
• Native vs. nfs-version mounts.

Needs coverage of Apache 2.x

Definitely to be labelled "Apache2/".

Needs coverage of Squid.

Because we all love sea-food. (Well, except for me.)

nginx can allow "worker_processes auto;"

Rather than specifying based on CPU-cores allow it to determine the best setting itself.

nginx example can be updated to show caching.

c.f. git.steve.org.uk

Needs cover of postgresql.

Potential issues here largely revolve around the "naive" vs. the "complex".

Almost certainly needs to be versioned:

• v8.4.x
• v9.1.x
• v9.2.x

couldn't build site

tried to test my PR by building via an install of a git clone of templer head. Comparing my generated index.html to yours, the columnar structure is missing, and the body of column 1 (Service and Server Optimization...) looks unprocessed. Likely there's some config or a templer plugin dependecy missing.

How to scale nginx worker_connections

https://github.com/skx/tweaked.io/blob/master/input/guide/nginx/index.wgn#L51 recommends increasing the amount of worker_connections but gives no guidance to the reasons.

Possible Pros:

• better to accept a connection quickly and handle it slowly, than rejecting the connection outright
• keep nginx from exhausting resources prematurely when talking to (slow) backends

Possible Cons:

• Increased memory usage
• nginx accepts more connections than it can handle, time-outing them after having committed resources to them, instead of rejecting them outright

Especially for the latter point, a(nother) reference to benchmarking might be appropriate.

Needs cover of MySQL.

Now I know a lot about MySQL, but distilling this is going to be a constant juggling act between really useful content which requires too much base-knowledge and simple things.

I'm not even sure how to start really.

/guide/kernel/ bad advice

net.ipv4.ip_local_port_range is not for servers but for clients, so change "For servers which are handling large numbers of concurent sessions"... to explain that it is for connecting to other servers (when you open many connections, not when you receive/accept connections).

net.ipv4.tcp_tw_recycle is very very bad, it is totally broken in kernel 4.10 and removed in kernel 4.12, but anyway in earlier kernels it was a very bad choice to use it, it breaks interoperability with many things.

net.ipv4.tcp_tw_reuse can add a 1 second delay to new outgoing connections so add that warning before enabling... it is better to have lots of TIME_WAIT, it is not bad at all to have many of them, TIME_WAIT is your friend not your enemy.

You have a good explanation here: https://vincent.bernat.im/en/blog/2014-tcp-time-wait-state-linux
Maybe add a link to it.

Thanks!

Layout needs work.

The current layout is nasty, and it should be improved.

But it should still be fast ;)

Mention timeouts on the webserver page.

Will come back to this later, but timeouts are important to cope with slow clients, and avoid contention issues.

Possibly cover: varnish.

Potential issues:

• 90% of varnish-value is the rules you write for your site.
• Two versions to cover 2.x + 3.x.

Using it at all is a major win, unless you're doing the naive pass-through.

On balance this could be added, but I suspect I won't write it.

kernel optimization - ec2 Instance fails Instance Status Checks after reboot

I have followed the kernel (http://tweaked.io/guide/kernel/) optimization guide for an m3.medium instance, but after rebooting the instance it fails instance status checks and can't connect to it over ssh.

Needs coverage of nginx-proxying

Operating nginx as a reverse proxy is almost entirely a different thing from running it as a webserver.

Cover reverse-proxying too.