Comments (5)
Google Cloud have some generic support for batch requests for their APIs. A solution for you could be to have a separate end-point for batch requests where you decode them into sub-requests:
- https://cloud.google.com/storage/docs/json_api/v1/how-tos/batch
- https://godoc.org/github.com/jfcote87/google-api-go-client/batch
This minimizes the number of HTTP requests, but not the load against the db.
Other good options for more specialized needs, involves writing RPC calls for your specific needs that operate against the same database as rest-layer. We have been doing this with success using the JSON RPC 2.0
spec and the zenrpc server-side library. JSON RPC also supports batch requests, and this can be enabled when relevant.
from rest-layer.
Thanks @smyrman for the speedy reply! I hope you are well.
Just a quick note, there is a difference between bulk
and batch
operations. Bulk
does repetitive action on a single resource, but batch
can do all kinds of actions against different resources.
Now, I have looked at Google way, but I view it as overkill for my needs at the moment. Also I would like to keep things rest-layer
way as possible, so no multi-part HTTP requests.
Also I guess it would be good if rest-layer
provides such(however limited) functionality out of the box.
Regarding your way of external operationagainst the same database
, I guess then you validate the request item against the schema
, update Etag
and updatedAt
fields to keep rest-layer consistency. It seems a lot of work. Aren't you working against the same rest-layer resources
?
from rest-layer.
Indeed, there is a difference between bulk and batch.
The JSON bodies you describe looks more like what you might want to do for BATCH; especially with multiple return statuses embedded in JSON. From a design perspective, supporting BATCH is very simpe, and very powerful.
Bulk howver, is very tricky, because the design of it is so tricky. If I where to implement patch for the list view, I think I would have implemented it as a single patch to apply to multiple resources matching a filter. This is tricky in rest-layer because the patch cannot be validated independently of what it's actually going to patch. If it is to contain a completely different request and response structure to everything else, then it's not a good fit.
Back to the RPC bit, then yeah, we parse from regular structs into a map[string]interface{}
and then use the resource layer directly, similar to how it's used by the rest-layer rest
package to validate the final insert/update.One huge benifit of this, turned out to be that structs are very easy to work with (and copy), so it was relativly easy to implement API versioning, header-based version selection and backwards compatibility. This is not something we are able to do for our REST API requests!
RPC is not intended for pure CRUD operations though; we use it mostly for M2M applications that are sporadically updated, and where backwards compatibility is key. However, because it's JSON, we are able to use JSON RPC in the front-end as well for everything that is a bit more complex than plain CRUD requests.
from rest-layer.
Thinking from another perspective, with HTTP/2 multiplexed connection, I don't see a benefit from supporting BATCH exclusively. One can just fire multiple concurrent HTTP requests, and the browser will multiplex them into a single already prepared connection.
I guess for BULK patching, you are talking patching against an arbitrary filter, where I am referring to patching against specific items, with theirs ID
specified.
In any case, I would prefer some generic approach, rather RPC mechanism outside rest-layer
.
Versioning I guess can be supported with multiple rest-layer
index-es, mounted on different routes, say /api/v1/
,/api/v2/
, etc.
I think for now I might just use multiple requests. Thanks @smyrman
from rest-layer.
For inspiration, here is another design for BULK (from jsonapi.org):
from rest-layer.
Related Issues (20)
- Validate fields depending on other field values HOT 2
- Remove old sub-resource syntax HOT 4
- go module: cannot find module providing package testing/mem HOT 3
- Explicit $eq in filter passes whole predicate to Validate(Query) HOT 3
- OnInsert/OnUpdate hooks can modify the new resource item, Etag not recalculated HOT 7
- `Prefer: return=minimal` can hide resource item on server modification HOT 12
- Consistent empty / null fields HOT 4
- Remove resource hooks in favor of resource middleware HOT 12
- Newest `zerolog` versions cause a panic HOT 4
- Bogus error line, when returning empty response on 204
- Etag should not depend on external state HOT 4
- Assist AutoIncrement FieldValidator with SQL backend HOT 8
- How use schema and resource package as standalone HOT 1
- Wrapping original issue in rest.Error HOT 7
- Feature Request: Cursor-based Pagination HOT 3
- pq: operator does not exist: uuid ~~ unknown HOT 7
- Filter on reference fields HOT 1
- Support for $not filter operation HOT 4
- Decouple ElemMatch from schema.Array struct
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rest-layer.