samsarahq / thunder Goto Github PK

When I run mutations on the /example server (e.g. addMessage or addReaction) using Insomnia as client, I always get the error "unknown field ..."

This is the mutation that I send

mutation {
  addMessage(text: "hi there")
}

Is there something wrong with the syntax (Insomnia can read the schema and gives me nice type-ahead suggestions) or does Thunder expect a different mutation style?

FieldFunc() should receive SelectionSet for particular node so that the resolver could build the appropriate query based on user's selection, but selectionSet is always nil.

Code example:

query.FieldFunc("viewer", func(ctx context.Context, selectionSet *graphql.SelectionSet) *User {
	// selectionSet is always nil
})

This is a pretty cool graphql approach.

From what i can see the SQL is also generated for mysql ?

Also i am really interested in if the system supports Asynchronous live query updates ?
For example say 100 clients are "subscribed" to a query, and 50 are online and 50 offline. Will it update the offline clients when they come back on live ?

Getting inconsistent results for a query including a case-insensitive match in a WHERE statement.

Assuming the following:

• name field in mysql DB is using mysql default utf8mb4 collation (which implies case-insensitive comparisons)
• the record described by relatedId and nameStr already exists
• using a context for which reactive.HasRerunner(ctx) returns true

err := ldb.QueryRow(ctx, &model, sqlgen.Filter{
	"related_id":   relatedId,
	"name": nameStr,
}, nil)
// err is sql.ErrNoRows and model is unpopulated


err := ldb.QueryRow(ctx, &model, sqlgen.Filter{
	"related_id":   relatedId,
}, sqlgen.SelectOptions
	Where:  "name = ?",
	Values: []interface{}{nameStr},
}, nil)
// err is nil and model contains appropriate data

reactive.HasRerunner(ctx) causes us to follow a code path utilizing reactive.Cache and in the first case leading to a call to db.batchfetch.Invoke(). In the second case we seem to bypass this and run db.QueryExecer() instead.

I'm unable to determine the exact cause of this due to a regression in golang 1.11 preventing variable inspection while debugging.

Info on golang regression causing inability to inspect variables:
go-delve/delve#1368
golang/go#27681

Hello, if using AWS IOT or grpc system, how to call thunder, use http handler? Can http handler be used with a websocket handler at the same service?, or can iot or grpc service update the database directly? Thank you.

Hey there ! First of all thanks for the amazing project !
I was wondering if you were planning on supporting the field description ? Even though I have no idea on how this could be achieved.

Thanks a lot !

I'm using a base struct for some common fields in my model, for example, an ID field. It seems when using a struct that has the base struct, those fields are not recognized by the library.

For example, this will not work:

type Base struct {
    ID string
}

type User struct {
    Base
    Email string
}

The query here fails with unknown field "id"

query {
    users {
        id
    }
}

As discussed in #70 (comment), thunder currently does not verify that non-null variables from operation definitions.

As we discussed irl, there is a new bug introduced in this PR that we think we are okay with ignoring for now.

That is, if a variable is marked optional in the operation definition, we do not check for whether or not it is passed in, which goes against the spec. We think this is okay because if the variable was actually required (i.e. used at some location), then the executor will get unhappy about it downstream.

For instance:

# query:
query Something($required: number!) {
  fieldWithOptionalId(id: $required)
}

# variables:
{}

This query will execute and return successfully in thunder today, when it should fail.

Note that a more severe case, failing to provide a variable that is required by a field, is caught by the executor:

# query:
query Something($required: number!) {
  fieldWithRequiredId(id: $required)
}

# variables:
{}

This query should fail as expected.

See #70.

Hi,

just wanted to point out a more idiomatic graphql library for go (if github.com/graphql-go/graphql doesn't fit you or not actively maintained).

https://github.com/neelance/graphql-go

thanks for open sourcing this library and the talk at graphql summit.

cheers,
bsr.

Is it part of the roadmap to support custom scalars

Hi, I am trying to compile the provided example for printing out the generated schema but it seems that introspection.ComputeSchemaJSON() does not accept *graphql.Schema

	server := &server{}
	schema := server.schema()
	introspection.AddIntrospectionToSchema(schema)
	valueJSON, err := introspection.ComputeSchemaJSON(*schema)
	if err != nil {
		panic(err)
	}

	fmt.Sprint(string(valueJSON))

results in

cannot use *schema (variable of type github.com/samsarahq/thunder/graphql.Schema) as github.com/samsarahq/thunder/graphql/schemabuilder.Schema value in argument to introspection.ComputeSchemaJSON

maybe codegen?

See #104.

https://snyk.io/vuln/golang:github.com%2Fsatori%2Fgo.uuid

Please consider migrating to github.com/gofrs/uuid which is a maintained fork.

I can't get Grapiql docs or auto-complete to work with more complex or lengthy schemas. When I attempt to start up graphiql, I get a load of code in red appearing in the middle frame:

r@http://192.168.1.220:3030/graphiql/bundle.js:6:3554 T@http://192.168.1.220:3030/graphiql/bundle.js:1:4094 getFields@http://192.168.1.220:3030/graphiql/bundle.js:1:8420 http://192.168.1.220:3030/graphiql/bundle.js:6:38934 o@http://192.168.1.220:3030/graphiql/bundle.js:6:39071 reduce@[native code] e@http://192.168.1.220:3030/graphiql/bundle.js:6:41559 i@http://192.168.1.220:3030/graphiql/bundle.js:20:297366 http://192.168.1.220:3030/graphiql/bundle.js:20:262685 promiseReactionJob@[native code]

Documentation waits forever. While queries do run, there's no auto-complete. Also, the prettify button will turn pink but does nothing.

Switch to https://github.com/golang-migrate/migrate

It'd be nice to have a per-connection cache that exists across subscriptions. Example where the current caching model does not help:

query Foo(author: "George Saunders") {
  books {
    name
    id
  }
}

query Foo {
  books(author: "George Saunders", genre: "nonfiction") {
    name
    id
  }
}

Even though the second query is a subset of the first, there is no thunder mechanism to hint to the backend that the previously fetched data can be reused.

Can you point me to the code that works with the replication log.

I am playing around with the idea to use another DB that natively support triggers.
I doubt you are interested in another Storage Driver ? I plan to use boltdb with raft and a dynamic sql like query language similar to spasql (https://en.wikipedia.org/wiki/SPARQL).

hello, In thunder's server.go code,
Const (
MaxSubscriptions = 200
MinRerunInterval = 1 * time.Second
)
The default setting MaxSubscriptions=200 and MinRerunInterval=5 second is a test value, or a parameter that is recommended for use in a production system? For real-time system services, the interval of 5 seconds will be longer. If we use the interval of 1 second, what will happen? In addition, MaxSubscriptions=200 is smaller for our users. If we set larger numbers, it should be The number of? ,Thank you.

Does anyone have an example on how to do authentication?

At the moment I was planning to use a Token, but it's Websockets, not sure if that will work or not.

Issuing a request with multiple queries does not return all the requested results with or without errors...

query {
  user(id: "myid") {
    id
    username
  }
  userList(first: 20) {
    edges {
      cursor
      node {
        id
        username
      }
    }
  }
}

Returns this:

{
  "data": null,
  "errors": [
    "user: record not found"
  ]
}

Should be more like:

{
  "data": {
    "userList": {
      "edges": [
        {
          "cursor": "bHVpc2pha29uMg==",
          "node": {
            "id": "70329254-b7b4-445e-a26f-e1de653bb1d0",
            "username": "user1"
          }
        }
      ]
    }
  },
  "errors": [
    "user.user: record not found"
  ]
}

My attempted fix...

http.go

package graphql

import (
	"context"
	"encoding/json"
	"errors"
	"fmt"
	"net/http"
	"sync"

	"github.com/samsarahq/thunder/batch"
	"github.com/samsarahq/thunder/diff"
	"github.com/samsarahq/thunder/reactive"

	"github.com/davecgh/go-spew/spew"
)

func HTTPHandler(schema *Schema, middlewares ...MiddlewareFunc) http.Handler {
	return &httpHandler{
		schema:      schema,
		middlewares: middlewares,
	}
}

type httpHandler struct {
	schema      *Schema
	middlewares []MiddlewareFunc
}

type httpPostBody struct {
	Query     string                 `json:"query"`
	Variables map[string]interface{} `json:"variables"`
}

type httpResponse struct {
	Data   interface{} `json:"data"`
	Errors []string    `json:"errors"`
}

func (h *httpHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	writeResponse := func(value interface{}, err error) {
		response := httpResponse{}
		if err != nil {
			response.Errors = []string{err.Error()}
		} else {
			response.Data = diff.StripKey(value)
		}

		responseJSON, err := json.Marshal(response)
		if err != nil {
			http.Error(w, err.Error(), http.StatusInternalServerError)
			return
		}

		http.Error(w, string(responseJSON), http.StatusOK)
	}

	if r.Method != "POST" {
		writeResponse(nil, errors.New("request must be a POST"))
		return
	}

	if r.Body == nil {
		writeResponse(nil, errors.New("request must include a query"))
		return
	}

	var params httpPostBody
	if err := json.NewDecoder(r.Body).Decode(&params); err != nil {
		writeResponse(nil, err)
		return
	}

	query, err := Parse(params.Query, params.Variables)
	if err != nil {
		writeResponse(nil, err)
		return
	}

	schema := h.schema.Query
	if query.Kind == "mutation" {
		schema = h.schema.Mutation
	}
	if err := PrepareQuery(schema, query.SelectionSet); err != nil {
		writeResponse(nil, err)
		return
	}

	var queries []*Query
	for _, s := range query.SelectionSet.Selections {
		q := &Query{
			Name:         s.Name,
			Kind:         query.Kind,
			SelectionSet: &SelectionSet{Selections: []*Selection{s}},
		}
		queries = append(queries, q)
	}

	var responses *syncMap = NewSyncMap()
	var errors *syncMap = NewSyncMap()

	collectResponse := func(selection string, data interface{}, err error) {
		spew.Printf("collectResponse.selection: %s\n", selection)
		if err != nil {
			errors.Store(selection, err.Error())
			return
		}

		if d, ok := data.(map[string]interface{}); ok {
			for key, val := range d {
				responses.Store(key, val)
			}
			return
		}

		responses.Store(selection, data)
	}

	finalizeResponse := func(value *syncMap, errors *syncMap) {
		response := httpResponse{
			Errors: errors.Errors(),
			Data:   diff.StripKey(value.internal),
		}

		responseJSON, err := json.Marshal(response)
		if err != nil {
			http.Error(w, err.Error(), http.StatusInternalServerError)
			return
		}

		http.Error(w, string(responseJSON), http.StatusOK)
	}

	var wg sync.WaitGroup
	wg.Add(len(queries))

	runners := make([]*reactive.Rerunner, len(queries))
	for i, q := range queries {

		e := Executor{}
		qry := q

		runner := reactive.NewRerunner(r.Context(), func(ctx context.Context) (interface{}, error) {
			defer wg.Done()

			ctx = batch.WithBatching(ctx)

			var middlewares []MiddlewareFunc
			middlewares = append(middlewares, h.middlewares...)
			middlewares = append(middlewares, func(input *ComputationInput, next MiddlewareNextFunc) *ComputationOutput {
				output := next(input)
				output.Current, output.Error = e.Execute(input.Ctx, schema, nil, input.ParsedQuery)
				return output
			})

			output := RunMiddlewares(middlewares, &ComputationInput{
				Ctx:         ctx,
				ParsedQuery: qry,
				Query:       params.Query,
				Variables:   params.Variables,
			})
			current, err := output.Current, output.Error

			if err != nil {
				if ErrorCause(err) == context.Canceled {
					return nil, err
				}

				collectResponse(qry.Name, nil, err)
				return nil, err
			}

			collectResponse(qry.Name, current, nil)
			return nil, nil
		}, DefaultMinRerunInterval)

		runners[i] = runner
	}

	wg.Wait()

	for _, runner := range runners {
		runner.Stop()
	}

	finalizeResponse(responses, errors)

}

sync_map.go

type syncMap struct {
	sync.RWMutex
	internal map[string]interface{}
}

func NewSyncMap() *syncMap {
	return &syncMap{
		internal: make(map[string]interface{}),
	}
}

func (m *syncMap) Load(key string) (value interface{}, ok bool) {
	m.RLock()
	result, ok := m.internal[key]
	m.RUnlock()
	return result, ok
}

func (m *syncMap) Delete(key string) {
	m.Lock()
	delete(m.internal, key)
	m.Unlock()
}

func (m *syncMap) Store(key string, value interface{}) {
	m.Lock()
	m.internal[key] = value
	m.Unlock()
}

func (m *syncMap) String() string {
	m.Lock()
	defer m.Unlock()
	result, err := json.Marshal(m.internal)
	if err != nil {
		return err.Error()
	}
	return string(result)
}

func (m syncMap) Error() string {
	return m.String()
}

func (m syncMap) Errors() []string {
	errors := []string{}
	for _, v := range m.internal {
		errors = append(errors, v.(string))
	}
	return errors
}

The current HTTPHandler Can only handle queries, it doesn't support mutation

I created a pull request that solves the issue #182

Can thunder support graphql fragment? How define the scheme file, thank you.

We need more documentation (outside of our changelog) to indicate that sql:",binary"/sql:",string"/sql:",json" do things (and what they do).

As far as I can tell, it's not possible to create input objects without the "_InputObject" suffix. I would much prefer to be able to create input objects with simply an "Input" suffix. This matches the convention of all of the other GraphQL systems we've built and interact with.

Is this currently possible?

It would make the most sense to me if input objects could optionally be explicitly registered in the same way as other objects:

schema.InputObject("FooInput", FooInput{})

This would also have the benefit of facilitating input object descriptions and potentially other features.

If computations pass float64s with these special values, we fail to serialize the output and close the socket here. JSON doesn't support these special floating point values.

yarn build
yarn run v1.5.1
$ rollup -c rollup.config.js && tsc -p ./tsconfig.json --emitDeclarationOnly

./src/index.ts → lib/index.js...
[!] (babel plugin) Error: It looks like your Babel configuration specifies a module transformer. Please disable it. See https://github.com/rollup/rollup-plugin-babel#configuring-babel for more information
src/index.ts
Error: It looks like your Babel configuration specifies a module transformer. Please disable it. See https://github.com/rollup/rollup-plugin-babel#configuring-babel for more information
at /Users/qi/TsProjects/thunder-react/node_modules/rollup-plugin-babel/dist/rollup-plugin-babel.cjs.js:59:105
at Object.transform$1 (/Users/qi/TsProjects/thunder-react/node_modules/rollup-plugin-babel/dist/rollup-plugin-babel.cjs.js:141:18)
at /Users/qi/TsProjects/thunder-react/node_modules/rollup/dist/rollup.js:20333:41
at
at process._tickCallback (internal/process/next_tick.js:160:7)
at Function.Module.runMain (module.js:703:11)
at startup (bootstrap_node.js:193:16)
at bootstrap_node.js:617:3

error An unexpected error occurred: "Command failed.
Exit code: 1
Command: sh
Arguments: -c rollup -c rollup.config.js && tsc -p ./tsconfig.json --emitDeclarationOnly
Directory: /Users/qi/TsProjects/thunder-react
Output:
".
info If you think this is a bug, please open a bug report with the information provided in "/Users/qi/TsProjects/thunder-react/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Thank you very much.

hello, is there any problem with the use of thunder-client Mutation? What is the correct way to use it? Because this comment was found in mutation.tsx, Thank you.

// This isn't quite right. Not sure how to unify the union
// type into a form that the conditional type will be okay with.

// Not sure how to unify these types. runMutation is combined
// at the argument level, while the external type is conditional.

Add the ability to perform all CRUD actions for multiple items

maybe we should remove valid field from the observer in graphiQL fetcher

Some ideas:

interfaces

• golang interfaces, with some annotation for SDL/field funcs

union types

• one-hot struct

When using Pagination, the graphql.HTTPHandler() leaks an additional "__key: ..." field on the resulting edges which is not wanted.

Current quickfix: use diff.StripKey(value) on the resulting value...

func (h *httpHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	writeResponse := func(value interface{}, err error) {
		response := httpResponse{}
		if err != nil {
			response.Errors = []string{err.Error()}
		} else {
			response.Data = diff.StripKey(value)
		}

      ...

Not sure if this is the best solution, but for the moment it suffices.

Any help on a permanent fix would be greatly appreciated.

Many thanks for putting together this package.

Cheers

Is it possible to marshal results using json tags. I cannot find a way to send graphql results without certain struct elements getting ignored using tags.

type NodeOut struct {
  ID        string `json:"id"`
  Name      string `json:"name"`
  CreatedAt *time.Time `json:"-"` 
}

Is there an undocumented way to do this? Many thanks!

It's not possible to create a schema with optional array (slice). So far I've been using pointers to let my struct fields optional and works pretty well. I had to create a new struct to encapsulate my slice in order to make it optional. There is any way to make it directly?

How do we specify that a field is of type ID instead of string or int. This helps support clients like relay.

Hey Guys, I am quite interested in the graphql subsystem it's quite a pragmatic implementation. I really like how you can mix in FieldFunctions. My use case has 2 sides 1) building a standard graphiql/graphql/relay frontend for managing data storage and 2) building a GRPC query api for bulk transfers.

Would it be possible to move the web socket / subscription stuff to another package and perhaps A standard post based relay example with graphiql would be fantastic.

&sqlgen.SelectOptions Query condition without between，Where restriction leads to too slow query of big data. Is there any other way?

Using browser tools you can see that /graphiql/bundle.js has port 3030 hardcoded in new u.Connection("ws://localhost:3030/graphql").

The readme doesn't specify this and it took me a while to figure out why graphiql's auto-complete wasn't working as I was using a port other than 3030 for my HTTP server.

Seems nice to somehow expose the underlying error. maybe we can replace path errors withs something like oops?

https://github.com/samsarahq/go/tree/master/oops

cc/ @berfarah

Would this work ?

You have 2 golang projects with their go types.

You have one DB wrapped by the thunder code.

• The binlog would have to be split out to each golang project correctly so they get the updates they are responsible for ?

Thanks for a great library. I have a couple of questions / suggestions though:

Is there a way to build ‘temporary schemas’ against a map[string]interface{} - this would allow me to create a self-discovery handler against an external REST API or back-to-back against DGraph etc?

Also, is there a plan to allow the use of other DBs such as SQLite or Postgres? Sort of linked to the first question as I’m developing a service to link to Graph and SQL databases and a fixed structure means a code change for a field change.

Thanks again for a great library!

Hello,

I'm new to this library so please excuse me if I'm out of scope or didn't understand something.

As far as I've understood, GraphQL errors are returned as strings :

• "name.something: error message" when they are path errors (ie. we gave a name to the query and declared them with errors.New in resolvers)
• "error message" : when we use a sanitized error or if we didn't declare a name for the object.

But the GraphQL specification is quite different : http://facebook.github.io/graphql/June2018/#sec-Errors

Errors messages should look like :

{
  "message": "Name for character with ID 1002 could not be fetched.",
  "locations": [ { "line": 6, "column": 7 } ],
  "path": [ "hero", "heroFriends", 1, "name" ]
}

This leads to issues with client libraries expecting the right format.

Tell me if I missed something. If so, I'm sorry.

In case I'm right, is there any interest in fixing that (which would probably be a breaking change for existing users, and Samsara in the first place) ? If so, we could probably contribute to it as we're definitely interested in using this library in our products (everything else is just perfect ;)).

It would be nice to model the query/variable parse step as middleware so that it's in the same execution codepath for error handling, downstream middleware, etc.

"This should be generic" - @stephen