matthewmueller / joy

A delightful Go to Javascript compiler (ON HOLD)

⚠️ Joy development is currently on hold. Recent Go -> WASM developments look like a more promising path to Go being a great choice for frontend development. I'm thrilled to see development moving so quickly now and I think our efforts are better spent improving the Go to WASM compiler. As for Joy, there are quite a few interesting nuggets of code in here that I'd encourage you to dig up, polish and open source 😊

Thanks for your interest in this project!

❤️ Matt

---

Translate idiomatic Go into concise Javascript that works in every browser. Use Go's type system and world-class tooling to build large web applications with confidence.

Visit mat.tm/joy to learn more about Joy.

---

Getting Started     ·    Examples     ·    Using the CLI

Contributing    ·    FAQ    ·    More Links

Joy API    ·    DOM API (coming soon!)    ·    Virtual DOM API (coming soon!)

---

Getting Started

1. Install Joy:

curl -sfL https://raw.githubusercontent.com/matthewmueller/joy/master/install.sh | sh

Note: you can also download from the releases tab.

2. Create a main.go file with the following code:

package main

func main() {
  println("hi world!")
}

3. Run the code in a real browser:

joy run main.go

4. See the compiled Javascript:

joy main.go

Examples

Visit https://mat.tm/joy/#examples or peruse the test suite.

Using the CLI

Compile Go into Javascript:

joy <main.go>

Compile and run the Go code in headless chrome:

joy run <main.go>

Build a development version of the code:

joy build --dev <main.go>...

Build a production version of the code (coming soon!):

joy build <main.go>...

Start a development server with livereload:

joy serve <main.go>...

Run joy help for additional details.

Contributing

So happy to hear you're interested in contributing! Here's a quick rundown of how to get setup:

Setup

1. Make sure you have the Go environment setup on your computer. There are quite a few better resources online on how to do that

2. go get -u -v github.com/matthewmueller/joy/... to install the compiler from source

3. go test -v to run all the tests

Links and tips:

• https://astexplorer.net/: I've been using this to figure out how to build the JS tree
• http://goast.yuroyoro.net/ : Simple Go AST viewer
• https://github.com/estree/estree/blob/master/es5.md: Link to the ES3 AST format, this is implemented in syntax.go
• https://golang.org/ref/spec: You may need to refer to this to see what types are possible in Go's AST
• To run all tests: go test -v
• To run individual tests: go test -v -run Test/08
• pretty.Println(ast) will pretty print the JS AST (requires this package)
• ast.Print(nil, node) will pretty print the Go AST

If you have any further questions, open an issue or reach out to me on twitter.

FAQ

Visit https://mat.tm/joy/#faq to view the FAQ.

More Links

• Run joy help to see what else Joy can do for you
• Visit mat.tm/joy to read more about Joy's origins
• Chat with us in #joy-compiler on Slack at gophers.slack.com
• Star github.com/matthewmueller/joy to follow the development
• Follow twitter.com/mattmueller for project updates

joy's Issues

go get cmd/... fail

$ go get -u github.com/matthewmueller/joy/cmd/...
# github.com/matthewmueller/joy/internal/cli
github.com/matthewmueller/joy/internal/cli/cli.go:37:14: stats.Client.Set undefined (type *analytics.Analytics has no field or method Set)

deep interfaces

This doesn't quite work yet:

package main

// VNode interface
type VNode interface {
	String() string
}

// Component interface
type Component interface {
	Render() string
}

type vnode struct {
	c *Component
}

func (v *vnode) String() string {
	return "earth"
}

func newComponent(c Component) VNode {
	return &vnode{&c}
}

type index struct {
	location string
}

func (i *index) Render() string {
	return i.location
}

func newIndex(location string) VNode {
	return newComponent(&index{location})
}

func main() {
	c := newIndex("mars")
	println(c.String())
}

remove packages that just contain files

e.g.

var preact = js.RawFile("./preact.js")

// Render the component
func Render(component jsx.Component) {
	js.Rewrite("$1.render($2, document.body)", preact, component)
}

Turns into:

(function() {
	var preact = js.RawFile("./preact.js")
	return {}
})()

Make `js.Rewrite()` more powerful

It's looking like js.Rewrite(...) is the right abstraction for writing definitions, but right now it's pretty limited, if we encounter a js.Rewrite() the entire function is replaced with just the contents of the rewrite.

It'd be great to be able to depend on external libraries (that may also be rewritten) as well as include multiple rewrites in a single function

fix zero values for pointers to structs and interfaces

should be nil

Improve range support

// Range expression                              1st value          2nd value
// [x] array or slice  a  [n]E, *[n]E, or []E    index    i  int    a[i]       E
// [ ] string          s  string type            index    i  int    see below  rune
// [ ] map             m  map[K]V                key      k  K      m[k]       V
// [ ] channel         c  chan E, <-chan E       element  e  E

• range over maps
• range over strings
• range over channels

Support: iota

this will go into golang/variables

recursive rewrites

func poss(promise interface{}) {
	js.Rewrite(`await (async function() {
 try {
	var res = await $1
	return [ res, null ]
 } catch (err) {
	return [ null, err ]
 }
})()`)
}

// Fetch fn
func Fetch(url string, options *Options) (*Response, error) {
	poss(js.Rewrite("unfetch($1, $2)", url, options))
	return nil, nil
}

This should enable quite terse bindings to be developed

Support: continue

Code Chunking and Hot Module Reload

Let's investigate things like building a non main package to be able to code chunk common libraries and do hot package swap.

Finish `joy build` for production builds

Right now golly doesn't produce ES3 code, to get it to output pure ES3 code:

• goroutines/channels or async/await to while and switch statements transform (what regenerator does)
• Promise polyfill: this could be supplied by the runtime, but wouldn't be necessary if we don't use async/await
• Replace fn.bind() calls with a runtime polyfill

Support: gh, ok := v.(Github)

Prettier Javascript with js.Raw & js.Rewrite

Right now we're prettying all our AST nodes, but for raw macros, they can't be as easily prettified without parsing them.

We may want to do that parsing in the future anyway though to fail early when you're writing invalid bindings.

Proper variable handling

Proper variable handling is done in golang/variable/variable.go but it needs to be tweaked and integrated into the translator.

Rename to jolly?

I like the word jolly a bit more, I was also thinking a component monorepo could be called jollies, which i like.

Golly is more unique than jolly though. Which name do you like more?

Document difference from GopherJS

First off, I'm very happy to see someone having another go at this. I'm very impressed by what I've seen.

That said, most people will ask themselves how this project differs from GopherJS.
I've been using GopherJS for a while in production now and don't feel like it is missing anything fundamental.
Could you clarify...

• ...why I may want to choose Joy over GopherJS, now or in the future?
• ...why you decided to start from scratch?
• ...if you see a shared future for Joy and GopherJS

The answers probably belong into the readme and FAQ, since this is probably every new visitors first question.

make topological sort sound

I haven't figured out a test case for this yet, but if there are cycles, despite sorting, I think there's a way for it to not be deterministic, or be deterministic and always fail (for dependencies that are cyclical via interfaces).

testdata/47-circular seems to be working, but I think if we reverse the order, we'll run into problems.

The solution is to walk the graph, reverse the edge direction, then remove any edges that are the "IMPLEMENTS" kind IF they have other edges coming from them, otherwise leave them in.

handle methods with the same name as the function

package main

func main() {
	items := test("a", "b", "c", "d")
	for _, item := range items {
		println(item)
	}

	a := aa{}
	items = a.test("a", "b", "c", "d")
	for _, item := range items {
		println(item)
	}
}

func test(a string, items ...string) []string {
	return items
}

type aa struct{}

func (*aa) test(a string, items ...string) []string {
	return items
}

Improve error handling

Right now errors are quite hard to decipher. Would be great to clean up the translator's error handling as well as better errors when you try compiling code that contains unsupported stdlib methods.

Support: User{"matt"}

Initializing structs require keys currently, but shouldn't have to

Paying the bills

This is a placeholder issue for all things related to self-sustainability.

Right now Joy was entirely funded and developed by me. As an independent developer with limited resources, I can only take this project so far on my own. The only way for this project to reach it’s full potential is for Joy to become community-driven and financially self-sustainable.

If you guys have ideas on how to get there, please let me know!

Hello world does not compile

Hello, I followed the instructions from running the joy command the first time and get this error message:

cannot find package "../../../matt/go/src/github.com/matthewmueller/joy/internal/runtime" in:
	/Users/matt/go/src/github.com/matthewmueller/joy/internal/runtime
   ⨯ error building packages: parse error: load error: unable to load the go package: couldn't load packages due to errors: ../../../matt/go/src/github.com/matthewmueller/joy/internal/runtime

Support: switch statements

Support rewrites with files as inputs

package preact

var preact = js.RawFile('./preact.js')

func Render(com component.Component) {
  js.Rewrite("$1.render($2)", preact, com)
}

package main

func main() {
  com := getComponent()
  preact.Render(com)
}

Results in (approx):

(function() {
  var preact = pkg["..."]
  var com = getComponent()
  preact.render(com)
})

Support: const

this will go into golang/variables

Support: labels

goto label

Compile to object literals when struct definitions don't have any functions attached to them

type Github struct {
  Name string `js:"name"`
}

func main() {
  println(Github{
    Name: "matt",
  })
}

Should compile to:

function main() {
  console.log({
    name: "matt",
  })
}

But right now it compiles to: new Github({ name: "matt" })

Handle embedded struct inheritance

type App struct {
  http.Handler
}

app := App{}
app.ServeHTTP

Haven't used this much personally yet, but should work with interfaces too.

Implement: golly test ./...

golly run input.go will already spawn a headless chrome browser and go test ./... in the golly repo uses headless chrome to run its tests.

It'd be so cool to have golly test ./... that we could use to test our frontend using a real browser the Go way.

jsast/api: should return errors

we should make this a bit higher-level and this is where additional typechecking will occur which may cause errors.

V2 architecture

have a pretty good idea of how the code ought to flow now, so i want to cleanup the architecture a bit to make it easier to add more complex features like interfaces without as much code smell. this work will be done in: #38

collect emails from CLI

Eliminate unuses struct values

This would be a nice enhancement. For example, in Go the proper way to represent an "" tag would be something like this:

type A struct {
	Class string
	Href string
	Title string
}

However, having all the possible fields of an <a> tag is wasteful, we can do better than this by only including the properties we're going to use.

imports should add $ in front to avoid collision

var header = pkg['github.com/matthewmueller/golly/testdata/49-jsx/header']

to:

var $header = pkg['github.com/matthewmueller/golly/testdata/49-jsx/header']

in:

pkg['github.com/matthewmueller/golly/testdata/49-jsx'] = (function() {
  var h2 = pkg['github.com/matthewmueller/golly/testdata/49-jsx/h2']
  var header = pkg['github.com/matthewmueller/golly/testdata/49-jsx/header']
  var jsx = pkg['github.com/matthewmueller/golly/testdata/49-jsx/jsx']
  var text = pkg['github.com/matthewmueller/golly/testdata/49-jsx/text']
  function main() {
    var header = h(
      h2.H2,
      {
        class: 'hi'
      },
      [
        'yo!',
        h(
          header.Header,
          {
            Title: 'lol'
          },
          ['hi!']
        )
      ]
    )
    console.log(header.Render().String())
  }
  return {
    main: main
  }
})()

Sort out renaming

@tj @tejasmanohar

I've been struggling pretty hard on sorting out how to rename Go functions and structs at build time to their DOM versions. I've come across 3 types of rename transforms that we're going to need to perform.

1. Turning structs into global API calls and eliminate the invokation:

package main 

type Document struct {
  Body *Node
}

func main() {
  document := Document{}
  println(document.Body)
}

to:

function main() {
  console.log(document.body)
}

2. Rename functions into their Javascript camelcase counterparts

package main 

type Document struct {
  Body *Node
}

func (d *Document) GetElementByID(id string) {
   // ???
}

func main() {
  document := Document{}
  page := document.GetElementByID("page")
  println(page)
}

to:

function main() {
  var page = document.getElementById("page")
  console.log(page)
}

3. Have the ability to turn certain functions into assignment statements

package main 

type Document struct {
  Body *Node
}

func (d *Document) SetInnerHTML(html string) {
   // ???
}

func (d *Document) GetInnerHTML(html string) string {
   // ???
}

func main() {
  document := Document{}
  document.Body.SetInnerHTML("<h2>Hi world!</h2>")
  println(document.Body.GetInnerHTML)
}

to:

function main() {
  document.body.innerHTML = "<h2>Hi world!</h2>"
  console.log(document.body.innerHTML)
}

If we can solve this in a generic way, this approach could be applied to any Go library that we want to compile to JS.

I have a couple thoughts about how to solve some of these but you guys are much more familiar with Golang than me, so if you have any ideas, that would be hugeeeeeee.

Type assertions shouldn't add the declaration as a dependency

Since gh := v.(Github), just compiles down to var gh = v, we don't need to include the Github declaration, if it's not being included anywhere else.

Note that this will probably not be the case for gh, ok := v.(Github)

RawFile has a different relative path for golly build vs golly run

stringify := js.RawFile("../../node_modules/preact-render-to-string/dist/index.js")

support golly build cmd/run/run.go

Right now go list inside getMains() returns main|command-line-arguments for some reason.

We'll need to filter out paths that point to go files before running them through go list

Support: switch t := v.(type)

Support type switches

Support remaining built-in functions

• copy
• make (with a capacity)

Support: func init()

support the init function which run before all mains

Empty struct values to JS throwing undefined errors

Go does some pretty crazy stuff around initializing empty structs:

type Document {
  Body Node
}

type Node {
  InnerHTML string
}

func main() {
  doc := Document{}
  doc.Body.InnerHTML = "hi"
}

In Go this would be fine since Body is initialized but right now the compiler naively does:

function Document(o) {
  this.Body = o.Body
}
function Node(o) {
  this.InnerHTML = o.InnerHTML
}

function main() {
  var doc = new Document({})
  doc.Body.InnerHTML // error! InnerHTML called on undefined Body!
}

This example illustrates the need to map Document back to its type in order to get it's defined struct key values, which the compiler doesn't currently do.

Improve map support

• isset

v, isset := mymap["key"]
if !isset {

}

• error out on non-strings as keys

Essential libraries

A list of library definitions that I need to be productive. Feel free to add more libraries that you'd like to see here:

• json
• fetch
  • promise
  • xhr (for cross-browser compat)
• websocket
• time
• strings
• url
• querystring
• errors
• preact
  • basic dom operations
• sha256 (for AWS signatures)

For the direct DOM ones, I think it makes sense for the API to mirror how the DOM does it (even if it looks weird in Go). We can always build libraries on top of the DOM definitions that make it more idiomatic Go code. An example of this is fetch, where it returns a Promise. A promise is a weird construct to work with in Go, but we can introduce a channel implementation on top of that, or use channels in your application code.

Method aliasing doesn't work when implemented by interface:

type Component interface {
	Render() VNode
}

// js:"render"
func (h *Header) Render() VNode {
	return nil
}

We should probably:

• throw an error if we do the above
• add support for method aliasing on the interface

Sourcemaps

I (currently) don't really use sourcemaps too much, but it'd be cool to be able to map javascript errors to go source inside the devtools.

Support: recover statements

Support: deferred functions

defer func() {

}()

golly build <file.go> should work

Also support -o, same as go build

Support: spreads

• function spreads
• slice spreads