caddyserver / caddy Goto Github PK

The New() functions that are in each middleware package are remnants of the early prototype days. Since the middleware packages are general and New() would never be invoked by any program other than Caddy, the New() functions should be moved into a separate package.

I know @mholt is already working on a new way to create and import caddy's middlewares. But I'm here to open a discussion about something related to that.

I would like to leave here some statements (my opinion) for each topic.

Core middleware

• Each core middleware should have it's own repository on github.
• The core should, for default, import only essential and very common middlewares, example: gzip, browse, proxy, redirect, extension.
• Other non common middlewares could be installed using the middleware management command line tool.

Middleware management

• The source file that imports and registers all middlewares config/middleware.go, should have only importing and registering instructions. This will allow us to create a command-line tool for the middleware management, that only generates this source file.

Imagine: middleware that pulls the latest version of your site when you push it to GitHub... built right into your web server.

I have bigger plans for automating deployment, but would this be useful to anyone in the interim? Would anyone like to start working on it sooner than I can get around to it?

When specifying a server address in the Caddyfile, should 127.0.0.1 be interchangeable with localhost? Right now, you'd have to specify both in the Caddyfile for the clients to be able to use both when making requests. Sometimes that's annoying.

localhost isn't always the only hostname to map to 127.0.0.1. I do believe that localhost is always 127.0.0.1 but I'm pretty sure the other way around isn't always true.

https://github.com/grpc/grpc-go

It will provide language agnostics and a solid performance RPC base that looks like it will last a good while.

The IDL aspects will possibly supply transport agnostics too, but I have not dug deep enough on that yet

It would be nice and simplify setup for users on heroku / docker, to just listen on 0.0.0.0 by default.

As pointed out by @mholt this won't break anything for users expecting localhost, and they can choose to explicitly bind to localhost

It would also be nice to accept default host via cli, -host

It would be amazing if quic protocol was envisaged. I am currently playing around with this and its astounding for RTC style interaction.

The current c code in google chrome could be codegened to go perhaps.

I saw this constantly on campus but was unable to reproduce the error at home. But seeing traffic from this morning is causing this error more. This is a hard one to debug since we'd have to drop to the transport layer.

A hostname like [::1] without a port (which is like [::1]:1234) should parse properly.

Heroku assigns each app a dynamic port, we need a way to pass $PORT as the port caddy should listen on. The ideal solution may be to allow a user to pass a Caddyfile snippet from the command-line the way nginx does with it's -g flag. http://nginx.org/en/docs/switches.html

This is probably just an ordering issue somewhere, but it had me scratching my head for a couple of minutes.

Steps to reproduce:

1. Create the following files in a new directory:

   index.html:

   {{/* super secret information goes here */}}
   Public information goes here

   Caddyfile:

   localhost:2020
   template / .html
   

2. Run caddy

3. Visit "/" and "/index.html" and compare the results

Expected results:

• Both execute the template and display only "Public information goes here"

Actual results:

• "/" displays the source of the file, starting with {{/* (i.e. the template does not get executed)
• "/index.html" works as expected.

How awesome would it be if your web server kept a static page search index (periodically updated) for your site and rendered search results onto a custom HTML template? No need for external services and no hassle setting it up - just another directive in the Caddyfile. I can think of many times in the past I would have used something like this if it was built right into my web server.

Thoughts on this? Feel free if you want to give it a shot - just comment here. This could be a fun and really useful middleware to have.

At least at basicauth the behaviour for Path.Matches should be different.

/test matches /test2 .. But I think is not the desired behaviour.

https://github.com/jordic/caddy/blob/path_matcher_bug/middleware/basicauth/basicauth_test.go#L13-L59

For the basicauth middleware, /test should not match /test2 because they can be distinct folders. I don't know if this must be fixed, inside the auth module, or inside the middleware path.

I think that a better aprouch should be, to consider some possible paths, like...

/test matches /test /test/1 /test/ but it don't matches /test2 neither /test.txt

When fastcgi has a problem and returns and 500 internal server error, it downloads a .gz file which has the file contents "��������Internal Server Error". Someone reported that this was happening on caddyserver.com as well so I think the issue is related to gzip. https://news.ycombinator.com/item?id=9455468

Can't find this in documentation, there is no default index file like index.html.
The php preset for fastcgi index index.php isn't working, at least on Windows x86.

Thanks for this very nice project :)

It would be really great if Caddy can be used to handle emails. It would remove the need for a separate email server as setting up a email server is a royal PITA.

Right now, the markdown directive only lets you specify CSS and JS files. It would be really useful to be able to specify a full page template like you can with the browse middleware.

Markdown should also be expanded to support front matter that contains metadata, etc... it could be used to fill template values and render the page.

Both the log and errors packages create log files. When the logs get really big, then what?

A middleware that creates log files should watch the size of the file (either by file size or number of entries - preferably file size) and gzip it when it gets large, and then start a new file. It could then keep only the last ~5 or so files to conserve disk space, purging the oldest ones when a new one is created.

Since log management spans multiple middlewares, it might be best for each middleware that makes logs to have a field for some sort of log management policy. Perhaps this "LogPolicy" type could be defined in the helper middleware package, which is allowed to be used by the middlewares. A LogPolicy type could define how big to allow each file to become, what to do when a file is full, how many to keep, etc. It could also have methods on it which execute these policies on a log file that gets passed in.

The default policy should probably be a conservative one that will ensure the disk does not become full. We could later build in ways to customize the policy as time permits and needs arise.

Hi I'm wondering if it is possible that Caddy can forward the real ip address of the clients, like the X-Forwarded-For and X-Real_Ip? I would like to use the clients IP address in my rails app which Caddy is proxying to.

Thanks and great work!

It seems unwise to serve files from the same directory Caddy is running from.

• http://localhost:2015/caddy.exe downloads the server it self.
• CHANGES.txt and README.txt which are distributed with Caddy will also automatically be the in server root.
• Any backup or template caddyfiles will be public
• If plugins are added which would reside in a /plugin directory or such. They'll also be served by the server.

At the very least I think the default server root should be /root or something similar.

HTTP/2 is fast, HHVM instead of FastCGI is even faster. It would be nice to have a config/template to use HHVM.

Here are example configs for Apache/Nginx: https://github.com/facebook/hhvm/wiki/FastCGI

It appears that specifying a 0.0.0.0 (wildcard) host is a special case that the virtual host feature doesn't handle yet. For example, 0.0.0.0:8080 should respond to anything that talked to port 8080.

Caddy responds when specifying just a port (":8080") but it's an error response.

drone.io is free for open source projects.

It's a new great product about Golang. But I want to know how about the proformance with nginx?

It may be useful to allow the default port to be specified from a command line flag.

In preparation to move the New() (generator) functions out of middlewares (#3), the config needs to be refactored. Right now, the config package uses different parsing facilities than the generators do, but the ones provided to the generators are more convenient and powerful. It would be nice if all the parsing could be done by the same types.

The generator functions will need access to the entire config, I think, to access things like hostname, port, and actual address that the server is listening on. Without refactoring, this change would result in circular imports.

Finally, this refactoring should allow the middleware packages to share a test harness to be more conveniently tested. It will also allow the generator functions to be put under test in a consistent, convenient way.

We may have to introduce at least one new package to accomplish this.

These changes are completely internal, and no user-facing behavior will change.

Would be nice if there was a url for latest build.

Right now I see like

/0.5.0/download.tar.gz

would be nice to have

/latest/download.tar.gz

This would ensure the heroku buildpack and docker container can always get the latest code without having to constantly update the buildpacks etc....

Thoughts?

I played around with the markdown middleware:

Template (default.html):

<html>
  <head>
    <title>{{ .title }}</title>
  </head>
  <body>
    {{ .markdown }}
  </body>
</html>

Caddyfile:

0.0.0.0:8000

markdown / {
    ext .md
    template default default.html
}

errors error.log
log access.log

Test markdown file test_yaml.md:

  ---
  template: default
  variables:
  - title: Test
  ---

  # Headline
  Lorem ipsum dolor sit amet ...

Result when accessing http://localhost:8000/test_yaml.md is 500 Internal Server Error. There is an entry in error.log saying: [PANIC /test_yaml.md] src/github.com/mholt/caddy/middleware/markdown/process.go:66 - assignment to entry in nil map.

The other formats (JSON, TOML) work as intended.

Would be nice to be able to browse Caddy source using sourcegraph...

https://sourcegraph.com/github.com/mholt/caddy. (It should only take 30 seconds.)

Hey Matt - great work so far!

Is there a way for Caddy to use index files (e.g. index.php) within sub-directories?

E.g.
http://local.dev/ --> Caddy will correctly use /index.php
http://local.dev/myfolder/ --> Caddy does not use /myfolder/index.php
http://local.dev/myfolder/index.php --> Works correctly, but not ideal

local.dev:80 {
    root path/to/document/root
    fastcgi / 127.0.0.1:9000 php
}

Thanks for any guidance

I would like the ability to add and remove backends from a site hosted by Caddy. Is this possible today, if not any plans for the future?

Looking at https://travis-ci.org/mholt/caddy it seems a lot of middleware is currently untested?

Is this true?

I am currently learning go, and this could be a good way to cut my teeth.

Could you add the repo for the documentation site? This would open it up to contributors as well.

Great job on Caddy, btw! Looks awesome.

Hello,

I cannot find any information about how to configure timeout, or max body size for an http request. These settings are quite important to avoid a DOS attack.

To make redirects and rewrites more powerful, like mod_rewrite.

could be good if there are some examples in tutorial for integrating the existing golang web framework like echo or martini

I tried using backslashes in my startup directive but it appears they might need escaped?
startup "C:\php\php-cgi.exe -b 127.0.0.1:9123 -c C:\php\php.ini" &

Error:
exec: "C:phpphp-cgi.exe": file does not exist

Seems like unexpected behavior to me but if it must remain it should definitely be documented.

I have a question.

Is it possible to host multiple sites in subdirectories?

www.XYZ.com
www.XYZ.com/blog
www.XYZ.com/owncloud
www.XYZ.com/.....

Caddy needs a cache middleware.

I'll be working on something basic this week. Probably just a basic LRU replacement algorithm and a configurable expiration time. In fact, if the gain is significant in nearly all cases, maybe it should be enabled by default... (a rare exception to the rule that all middleware need to be enabled explicitly using the Caddyfile).

Continuation from discussion at end of issue #13. /cc @darron

I created a Docker image for Caddy - https://github.com/darron/caddy-docker - but in order to do that and make a really small image I needed to statically link the final binary. The final Docker image was 23MB - vs using Ubuntu as a base image which was 148MB.

If possible, using your binaries would be better - people trust the author more than some random person compiling it - but you'd need to build them a little differently and statically link them.

Would that be possible for you to look at doing?

...

CGO_ENABLED=0 go build -a -installsuffix cgo

-installsuffix doesn't appear to have any effect on go build (without -i, anyway)?

I'm using gox to do the builds so I'll have to experiment. But this doesn't fail, at least: CGO_ENABLED=0 gox github.com/mholt/caddy - if I sent you a build from this would you try it to see if it works? And if so I'll make it part of the automated build process.

I know there is support for FastCGI, but could simple CGI be supported as well ?
I am enthusiastic about the performance of Caddy (wow!), but the lack of CGI support makes it not suitable for my needs (my company is using Progress Webspeed Messenger to communicate with our data server).

As for the directive, it could be like that for example:

cgi /cgi-bin/ {
    ext   .sh
    split .sh
    index script.sh
}

Don't know if that's possible :-) Just suggesting

By default crypto/tls use TLS1.0 as min_version protocol supported, and if not specified, all ciphers implemented in https://github.com/golang/go/blob/master/src/crypto/tls/cipher_suites.go#L75-L93

For compatibility maybe we can set min_version to SSL3, but accept both SSLv3 and TLS1.0 has security issues, like the POODLE attack (https://www.openssl.org/~bodo/ssl-poodle.pdf)

Accept all ciphers also have security problems. Mozilla opsec team recommends this configuration for Go crypto/tls: https://wiki.mozilla.org/Security/Server_Side_TLS#Go
HTTP2 also has some ciphers in blacklist: https://github.com/bradfitz/http2/blob/b6255645465a25b25f804acb9b3a54009e80c2a4/server.go#L281-L302

The question here is: Caddy accept all ciphers and protocols by default, and turns this configurable in caddyfile, or assume the security position for users and don't accept weak ciphers and insecure protocols?

I've tried to get a php Hello World running but all I get is a 502 Bad Gateway. Markdown does actually work.

Caddyfile

localhost:2020
markdown /md
fastcgi /php php
log access.log
errors error.log

Inside /php I have index.php with html and php.

Access log is filling nice when navigating to /md.
Error log is empty when the 502 occurs.

[edit]
Or am I understanding it wrong and Caddy is only the proxy to the fastcgi server?
[/edit]

Hi, it would be prudent to remove gzip from the default quick start guide as it is vulnerable to this, more information and discussion here.

Hi,

I have a simple server listening on port 9000:

package main

import (
    "fmt"
    "net/http"
)

func testHandler(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "test\n")
}

func main() {
    http.HandleFunc("/test", testHandler)
    http.ListenAndServe(":9000", nil)
}

Caddy forwards requests and gzips responses:

0.0.0.0:8000
proxy /test http://localhost:9000
gzip

For the URL http://localhost:8000/test Chrome says ERR_CONTENT_LENGTH_MISMATCH.

$ curl -v localhost:9000/test
* Connected to localhost (127.0.0.1) port 9000 (#0)
> GET /test HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost:9000
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Sat, 02 May 2015 10:25:03 GMT
< Content-Length: 5
< Content-Type: text/plain; charset=utf-8
<
test

$ curl -v -H "Accept-Encoding: gzip" localhost:8000/test
* Connected to localhost (127.0.0.1) port 8000 (#0)
> GET /test HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost:8000
> Accept: */*
> Accept-Encoding: gzip
>
< HTTP/1.1 200 OK
< Content-Encoding: gzip
< Content-Length: 5
< Content-Type: text/plain; charset=utf-8
< Date: Sat, 02 May 2015 10:25:24 GMT
* Server Caddy is not blacklisted
< Server: Caddy
<
* transfer closed with 5 bytes remaining to read
* Closing connection 0
curl: (18) transfer closed with 5 bytes remaining to read

It looks like the Content-Length header value from the backend response (proxy middleware) is not adapted by the gzip middleware. In this case (string test\n) the Content-Length should be > 30 bytes.

To make proxy more powerful,like ngnix:
proxy_set_header Custom-Header "my custom"

Hi, how about a plugin system via pingo or lua?

Right now you can only add and overwrite headers using header / Key Value.

What if I want to remove a header? Can we add something that will remove header e.g. -header / Server so I can remove headers that are being sent by server or application?

Line in question would probably be: https://github.com/mholt/caddy/blob/master/middleware/headers/headers.go#L25

Function in question would probably be: http://golang.org/pkg/net/http/#Header.Del

I could do it myself but I have no experience in Go and would probably not write it very pretty.