hoisie / web

The easiest way to create web applications with Go

web.go

web.go is the simplest way to write web applications in the Go programming language. It's ideal for writing simple, performant backend web services.

Overview

web.go should be familiar to people who've developed websites with higher-level web frameworks like sinatra or web.py. It is designed to be a lightweight web framework that doesn't impose any scaffolding on the user. Some features include:

• Routing to url handlers based on regular expressions
• Secure cookies
• Support for fastcgi and scgi
• Web applications are compiled to native code. This means very fast execution and page render speed
• Efficiently serving static files

Installation

Make sure you have the a working Go environment. See the install instructions. web.go targets the Go release branch.

To install web.go, simply run:

go get github.com/hoisie/web

To compile it from source:

git clone git://github.com/hoisie/web.git
cd web && go build

Example

package main
    
import (
    "github.com/hoisie/web"
)
    
func hello(val string) string { return "hello " + val } 
    
func main() {
    web.Get("/(.*)", hello)
    web.Run("0.0.0.0:9999")
}

To run the application, put the code in a file called hello.go and run:

go run hello.go

You can point your browser to http://localhost:9999/world .

Getting parameters

Route handlers may contain a pointer to web.Context as their first parameter. This variable serves many purposes -- it contains information about the request, and it provides methods to control the http connection. For instance, to iterate over the web parameters, either from the URL of a GET request, or the form data of a POST request, you can access ctx.Params, which is a map[string]string:

package main

import (
    "github.com/hoisie/web"
)
    
func hello(ctx *web.Context, val string) { 
    for k,v := range ctx.Params {
		println(k, v)
	}
}   
    
func main() {
    web.Get("/(.*)", hello)
    web.Run("0.0.0.0:9999")
}

In this example, if you visit http://localhost:9999/?a=1&b=2, you'll see the following printed out in the terminal:

a 1
b 2

Documentation

API docs are hosted at https://hoisie.github.io/web/

If you use web.go, I'd greatly appreciate a quick message about what you're building with it. This will help me get a sense of usage patterns, and helps me focus development efforts on features that people will actually use.

About

web.go was written by Michael Hoisie

web's Issues

Why is static a special case?

In the code you special-case "/static/" but I think there should rather be a "static handler" that can be attached to routes like everything else. Maybe that's naive, but it seems weird looking at the code to have that special case.

Error during build

When doing a build using 8g / 386 I see this:

8g -o go.8 fcgi.go request.go scgi.go servefile.go status.go web.go
fcgi.go:6: syntax error: unexpected :, expecting }
make: *** [go.8] Error 1

build error - "request.go:75: cannot use hr.Header (type http.Header) as type map[string] string in field value"

Currently getting this error if I try and build (either direct from git or using goinstall).

make
6g -o go.6 fcgi.go request.go scgi.go servefile.go status.go web.go
request.go:75: cannot use hr.Header (type http.Header) as type map[string] string in field value

I'm running latest go build on Ubuntu 10.10 64bit.

Serving static files with scgi doesn't work

Add secure cookies

This is important. Cookies are easily forged, so securing cookies is the right approach.

Any back end supported...?

can we access databases like mysql or oracle using go programming language..?

Add a "webgo clean" command

Test nginx/proxying

Add cookie-based sessions, including session storage

Cookies (by default) should expire 30 minutes after the last request

Add support for multiple headers with the same name

As an example, multiple Set-Cookie headers

support gzip encoding

When a request has Accept-Encoding: compress, gzip, it should encode the content as gzip
Also, add a Vary: Accept-Encoding to the response header

Add a way to alter the response headers

cookie age hardcoded

web.go line 78

The cookie age parameter is ignored. The cookie expiration is hard-coded for 30 minutes.

I think the SetCookie function should be replaced with this:

//Sets a cookie -- duration is the amount of time in seconds. 0 = forever
func (ctx *Context) SetCookie(name string, value string, age int64) {
    var utctime *time.Time
    if age == 0 {
        // 2^31 - 1 seconds (roughly 2038)
        utctime = time.SecondsToUTC(2147483647)
    } else {
        utctime = time.SecondsToUTC(time.UTC().Seconds() + age)
    }
    cookie := fmt.Sprintf("%s=%s; expires=%s", name, value, webTime(utctime))
    ctx.SetHeader("Set-Cookie", cookie, false)
}

I dug around in the HTTP RFC and googled my heart out and I couldn't find the max allowed value of "expires". I figure 2^31 - 1 (sometime in 2038) is a reasonable value of "forever".

BTW web.go is awesome! :-)

fcgi: bug in "case FcgiStdin"

It is not that uncommon to have a POST body large enough to get split into multiple FCGI_STDIN chunks, so this should just be appending to the buffer, rather than assigning it.

case FcgiStdin:
if h.ContentLength > 0 {
var buf bytes.Buffer
buf.Write(content)
req.Body = &buf

And despite the spec not being very clear on what FCGI_DATA is used for, browsers do use it; so it should probably be handled the same way.

[ Great framework, btw, keep improving it :) ]

Add support for fcgi

filename of uploaded file is not accessable

in commit 0554b31
you changed the way in which the filedate is initialized:

-                    r.Files[name] = filedata{filename, rest}
+                if params["filename"] != "" {
+                    r.Files[name] = filedata{name, data}

and the result is that the filename is no longer available:
the Filename field of filedata now contains the value of the 'name' attribute of the file input form field.

I solved this for me by changing it back as follows

-                     r.Files[name] = filedata{name, data}
+                     r.Files[name] = filedata{params["filename"], data}

web.SetCookieSecret does not exist

Hello,

web.SetCookieSecret is mentioned in web.go line 103
ctx.Logger.Println("Secret Key for secure cookies has not been set. Please call web.SetCookieSecret")
but does not exist in web.go.

Add api docs and sample applications to getwebgo.com

Add RemoteAddr to ctx.Request

Information about client remote address is required for some class of web applications.

Compiling web.go on Arch Linux

Hi,

I am using a version of Go, but I don't know which, (man 6c, go -v or go --version does not work on my system). The package version is, "2010_09_29-1", however.

When I pull web.go from git and try to compile it, I get:

make
6g -o go.6 fcgi.go request.go scgi.go servefile.go status.go web.go
fcgi.go:244: s.Logger.Println undefined (type log.Logger has no field or method Println)
fcgi.go:299: s.Logger.Println undefined (type log.Logger has no field or method Println)
fcgi.go:305: s.Logger.Println undefined (type log.Logger has no field or method Println)
scgi.go:148: s.Logger.Println undefined (type log.Logger has no field or method Println)
scgi.go:175: s.Logger.Println undefined (type log.Logger has no field or method Println)
scgi.go:182: s.Logger.Println undefined (type log.Logger has no field or method Println)
web.go:117: ctx.Server.Logger.Println undefined (type log.Logger has no field or method Println)
web.go:418: cannot use "" (type string) as type io.Writer in function argument:
string does not implement io.Writer (missing Write method)
web.go:418: cannot use log.Ldate (type int) as type string in function argument
web.go:418: not enough arguments to function call
web.go:117: too many errors
make: *** [go.6] Error 1

Replacing the log statements with fmt.Printf and fmt.Println makes it compile, but logging is more useful...

How to reproduce:
Just type "make".

Thanks.

Allow for setting of the cookie path

I find it useful to be able to specify the cookie path when setting the cookie. What if we have an extra argument to the SetCookie/SetSecureCookie functions...

Add support for cache-control headers

See the suggestions from redbot

Add helper methods that can get request parameters (for get and post)

Add a test harness

Get rid of Session

It's a hack right now -- maybe add a plugin for it down the line. As long as SetCookie works, that's good enough

Failed to build due to changes of http.ResponseWriter

web.go:229: c.conn.Hijack undefined (type http.ResponseWriter has no field or method Hijack)

Pls refer to : http://codereview.appspot.com/4245064/

Currently I have fixed the issue on my box follow the guide in above link. Hope you can include this change.

Thanks,
Cj

build error: vec.Data undefined

on tip of go and tip of web.go:

/home/ericvh/bin/6g -o go.6 fcgi.go request.go scgi.go servefile.go status.go web.go
request.go:129: vec.Data undefined (type vector.StringVector has no field Data)
make: *** [go.6] Error 1

design a middleware system for web.go

Lately I've been finding use cases for a middleware system. For example - generic cache-control handling and authentication

It will most likely follow the wsgi format -- something very simple for now

Figure out a way to control logging

Maybe have a logger object that defaults to stdout

implement an in-house template library

the pkg/template makes writing web templates difficult. web.go needs something better suited for the web, like mustache

Allow http.Handlers to be registered in web.go

If there were

web.Handle(route string, handler http.Handler)

or something similar, it would be possible to more easily port old web code that uses the standard http packge to web.go.

More interestingly, it would make it possible to register websockets handlers and other things that require lower level control.

log.Exit disappeared in Go release 2011-02-01.1

8g -o _go_.8 fcgi.go request.go scgi.go servefile.go status.go web.go 
web.go:449: undefined: log.Exit

web.go is not compatible with spawn-fcgi

Reported by Alexey Zilber:

Running a web.go app with spawn-fcgi doesn't work:

sudo -u lighttpd spawn-fcgi -f /path/to/app -a localhost -p 8803

spawn-fcgi.c.230: child exited with: 0

In this case, as far as I know, web.go should read from stdin and write to stdout

Monitor template and static directories for file changes

Try to get http.statusTexts exported

You should try to get a CL into the Go tree approved so you don't have to copy the status messages. DRY and all that. :-D I can do it if you don't have time for the process. Of course they could turn us down... :-/

Routes are tested in reverse of intuitive order

One URL may match many regexes. The order in which route regexes are tested is therefore significant. This order, in my experience, is "top to bottom" -- this is present in Django, and in web.py from which web.go is vaguely inspired.

Thus, I would expect a call to http://localhost:9999/ to respond with the string first, given the following program:

package main

import (
  "web"
  )
func main() {
  web.Get("/", func() string { return "first" } )
  web.Get("/", func() string { return "second" } )
  web.Run("0.0.0.0:9999")
  }

However, it responds with second.

I only started Go today :\ -- but looking at http://github.com/hoisie/web.go/blob/master/web.go ,
the reason is that routes is a map rather than an array. routes is only used when iterating over it, suggesting the want of an array rather than map. Nor is the map key (a compiled regex) necessary as each route struct has the compiled regex inside it. Am I right?

Enhancement: dynamic recompilation manager ... ?

This is (almost) my first experience with web development using a compiled language. I've become extremely used to being able to just change my code, and when I refresh the page, it runs differently. I do this literally more than once a minute.

Compiling a binary every time I want to see a change, even if compilation is almost instantaneous, is hard work. I don't think it's a fundamental issue with compiled languages, though -- how about if some scheme like the following was followed:

The system is aware of the source from which it is compiled, and tracks those files for changes. When a new HTTP request comes in, if changes to the source have been made, it dies, recompiles, starts again, and then handles the request. There'd be issues if you're storing application state in memory, but ... I don't.

I should say, this management would be done by another binary rather than the app itself. Let's call it runwebgo, and I can call it as with any other binary. And let's say for my application, called blog, I provide a plaintext file called .runwebgo, which contains:

• a list of relative paths to sourcecode files to watch
• a command to execute when recompilation is required (e.g. make)
• a port on which my running app will listen

Now, instead of running make and then ./blog every time, I just run runwebgo in the top level of my app directory. runwebgo then:

• parses my .runwebgo file for the list of files, the command, and the port
• watches all files for updates, setting a changed flag when a file changes
• starts up my app
• runs as a web server (in exactly the same way as a normal web.go app) listening on some odd port (other than the 9999 or 80 which I may have my app configured to)
• every time a request is received, it checks the changed flag. If things have changed, it kills my app, recompiles it, re-runs it. In either case, it then sends on the request to the port of my app.
• when it is killed, it kills my app as well.

It could also observe changes to the .runwebgo file itself, and dynamically update the source list if I resave it.

I imagine creating this would not be too hard or use anything outside the standard library. The upshot IMO is massive: I just run one command and leave it for my whole coding session.

fcgi support doesn't work on shared hosting services

Many shared hosting services (ie. dreamhost for me) don't allow you to specify an external fcgi by address:port and instead using fd 0 as the listen socket for the FastCGI. It would be cool if web.go supported this mode of operation unlocking relatively cheap hosting of go applications.

Value of Request.Params

• Why Request.Params has []string as value? I'm supposed that should be string. Anyway it could not get unicode characters, so to use []int would be a solution.

http://github.com/hoisie/web.go/blob/master/request.go#L32

• Would not better to return a type of data according to the field type? For example, for a checkbox
  
  can be used a boolean.

index.html in subdirectories of ./static

I have a subdirectory in my static directory which contains an index.html, however, requests on either /subdir or /subdir/ 404, only /subdir/index.html works. Is this intended behavior?

Add more tests

1. Make sure that setting http headers works
2. Make sure status codes are properly set (error and redirect)

have a way to generate different status codes (404, etc..) from handlers

web.go fails to build with latest release

goinstall github.com/hoisie/web.go
8g -o main.8 main.go resume.go /mnt/home/droundy/src/go/pkg/linux_386/github.com/hoisie/web.go.a
main.go:8: inconsistent definition for type reflect.FuncType during import
struct { reflect.commonType "func"; dotdotdot bool; in []_runtime.Type; out []_runtime.Type }

URL prefix

Many times you are going to use an URL prefix, like:

urlPrefix = "/wiki/"
web.Get(urlPrefix+"view/(.+)", viewHandler)
web.Post(urlPrefix+"save/(.+)", saveHandler)

so it would be helpful a function that stores that prefix and then were inserted at the string, i.e.

web.Prefix("/wiki/")
web.Get("view/(.+)", viewHandler)  // => /wiki/view/
web.Post("save/(.+)", saveHandler)  // => /wiki/save/

web.Prefix("/blog/")
web.Get("view/(.+)", viewHandler)  // => /blog/view/

// Prefix Cleaning
web.Prefix("")
web.Get("view/(.+)", viewHandler)  // => view/

Use struct tags match Params when using `UnmarshalParams`

To deal with the naming constraints of a struct, the json module allows one too specify tags to guide unmarshaling/marshaling:

type MyParams struct {
    FooBar string "foo_bar"
}

This would serialize the contents of FooBar with the key "foo_bar" and deserialize such that an item with the "foo_bar" will be placed into FooBar.

I added this modification to allow web.go use these tags to guide parameter unmarshaling in the same manner:

diff --git a/request.go b/request.go
index 0e2fc4c..9df8837 100644
--- a/request.go
+++ b/request.go
@@ -328,17 +328,33 @@ func (r *Request) writeToContainer(val reflect.Value) os.Error {
             v.SetElem(mk, mv)
         }
     case *reflect.StructValue:
+        // A little preprocessing to make this faster
+        tagMap := map[string](reflect.Value) {}
+        nameMap := map[string](reflect.Value) {}
+
+        ty := v.Type().(*reflect.StructType)
+        for i := v.NumField(); i >= 0; i-- {
+            field := ty.Field(i)
+            if len(field.Tag) > 0 {
+                tagMap[strings.ToLower(field.Tag)] = v.Field(i)
+            }
+
+            if len(field.Name) > 0 {
+                nameMap[strings.ToLower(field.Name)] = v.Field(i)
+            }
+        }
+
         for pk, pv := range r.Params {
-            //try case sensitive match
-            field := v.FieldByName(pk)
-            if field != nil {
+            // try struct tag matching first
+            if field, ok := tagMap[strings.ToLower(pk)]; ok {
                 writeTo(pv, field)
+                continue
             }

-            //try case insensitive matching
-            field = v.FieldByNameFunc(func(s string) bool { return matchName(pk, s) })
-            if field != nil {
+            // now try case-insensitive
+            if field, ok := nameMap[strings.ToLower(pk)]; ok {
                 writeTo(pv, field)
+                continue
             }

         }

Cannot compile: utc1.RFC1123 undefined

Programmation/Go/web.go % git pull
Already up-to-date.
Programmation/Go/web.go % make
/home/bortzmeyer/bin/8g -o go.8 fcgi.go request.go scgi.go servefile.go web.go
web.go:76: utc1.RFC1123 undefined (type time.Time has no field RFC1123)
make: *** [go.8] Error 1

Build fails under Debian Squeeze x64

... Following output:

:~/web.go$ gomake
6g -o _go_.6 fcgi.go request.go scgi.go servefile.go status.go web.go
scgi.go:106: cannot use &tmp (type *[1024]uint8) as type []uint8 in function argument
scgi.go:118: cannot use &tmp (type *[1024]uint8) as type []uint8 in function argument
servefile.go:75: cannot use &buf (type *[1024]uint8) as type []uint8 in function argument
make: *** [_go_.6] Error 1

GetSecureCookie format assumption

In the GetSecureCookie function if the cookie content doesn't fit the required format we can expect index out of range errors.

parts := strings.Split(cookie, "|", 3)
val := parts[0]
timestamp := parts[1]
sig := parts[2]

Websocket support

It would be nice to have websocket handler support:

web.Websocket("/test", websocketTest)

It's something I like in Mojolicious and would like to see in web.go.

Cast string arguments in request path to match handler types

For instance, the route "/([0-9]+), should be able to be passed to the following handler:

func handler ( n int ) string { ... }