vasanthv / jsonbox Goto Github PK
View Code? Open in Web Editor NEWHTTP-based JSON storage.
Home Page: https://jsonbox.io
License: MIT License
HTTP-based JSON storage.
Home Page: https://jsonbox.io
License: MIT License
Add a route to get meta information about a box.
A silly question: I'm curious that where will my data be stored ? @vasanthv
Is it just me, or are requests taking a long time to process? I'm noticing about 7-20 seconds to handle a fetch. It's not an issue with my computer or network.
Is there any alternative? I would prefer one with the same syntax of get / post, or any similar service, where I can store JSON data, storing it with a secret key, while they remain readable by ordinary AJAX call, and CORS enabled, of course.
I wanted to suggest replacing the current query syntax with a simpler one, the same as GitHub uses in its search queries. Here is a small example of how this might look:
# exact match
https://jsonbox.io/demobox_6d9e326c183fde7b?q=arya%20stark
# query with AND operator
https://jsonbox.io/demobox_6d9e326c183fde7b?q=dogs+cats
# query with OR operator
https://jsonbox.io/demobox_6d9e326c183fde7b?q=dogs|cats
# exact match with field name
https://jsonbox.io/demobox_6d9e326c183fde7b?q=name:arya%20stark
# exact match by multiple fields
https://jsonbox.io/demobox_6d9e326c183fde7b?q=name:arya%20stark+age:23
# greater than
https://jsonbox.io/demobox_6d9e326c183fde7b?q=likes:>100
# equal or greater than
https://jsonbox.io/demobox_6d9e326c183fde7b?q=stars:>=1000
# less than
https://jsonbox.io/demobox_6d9e326c183fde7b?q=posts:<100
# equal or less than
https://jsonbox.io/demobox_6d9e326c183fde7b?q=_createdOn:<=2019-09-10T09:23:01.105Z
You can find a few more examples from these links:
https://developer.github.com/v3/search/
https://help.github.com/en/articles/understanding-the-search-syntax
It would be useful to standardize and document the responses for each operation, such as the status code, response body, etc.
See https://developer.mozilla.org/en-US/docs/Web/HTTP/Status
Here are some suggested examples:
Code | Description | Uses |
---|---|---|
200 | OK | generic |
201 | Created | create requests |
Code | Description | Uses |
---|---|---|
400 | Bad request | generic |
401 | Unauthorized | when requesting protected boxes without API key |
404 | Not Found | when requesting invalid resources (ID, collection, etc.) |
405 | Method Not Allowed | e.g. when attempting to PUT a collection endpoint |
413 | Payload Too Large | not sure if you have a real 10KB limit, but would apply in that case |
429 | Too Many Requests | you don't state that you rate limit, but including this anyway |
Code | Description | Uses |
---|---|---|
500 | Internal Server Error | generic |
503 | Service Unavailable | I saw issues in the past with Mongo Atlas problems |
The response body would also benefit from being standardized with static or conditional fields. As a suggestion, you could provide the following standard response fields on the body:
status
(one of: "success", "fail", or "error")message
(when status
is "fail" or "error": a description of the error or the reason why the request failed)data
(when status
is "success"โmight not used in cases like DELETE
unless you return the deleted data in the response)Here's an example response for a PUT request with an invalid ID:
{
"status": "fail",
"message": "Invalid record ID"
}
And here's one for a GET request:
{
"status": "success",
"data": [
{
"_id": "5d776b75fd6d3d6cb1d45c52",
"name": "Daenerys Targaryen",
"age": 25,
"_createdOn": "2019-09-10T09:23:01.105Z"
},
{
"_id": "5d776b75fd6d3d6cb1d45c53",
"name": "Arya Stark",
"age": 16,
"_createdOn": "2019-09-10T09:23:01.105Z"
}
],
}
great idea, thanks!
My main limitation at the moment is that I cannot query inside an array. I cannot search for something that is contained in an array that is contained in an object.
One workaround could be removing the array from the object and placing it inside another collection with an external key, but that would make the client side code much more cumbersome.
Would this extension be something you are going to implement?
Calling delete on a non-existing id need not throw an error. In that way, the endpoint would be idempotent. i.e., calling it multiple times won't be an issue.
Hi @vasanthv,
thank you for this awesome service!
Do you plan to create Docker image and instructions on how to self-deploy jsonbox?
Best regards,
Herman
Need to recover automatically, when database goes down and comes back online.
Today I received feedback that when connecting to jsonbox, someone encountered Cloudflare man-machine verification and cause program error. Is it possible not to use Cloudflare? Otherwise, once the man-machine verification occurs, it will cause the data acquisition failure. This is very bad! Thanks.
I get this heroku application error whenever I try to get data from my jsonbox @vasanthv . I'm not sure what the issue is.
So that the "read the docs" link on https://jsonbox.io/ can point to the github page, which is easier to read.
Hi,
Thanks for this great service. Do you have any plans to enable CORS?
I'm getting the below error when querying from browser:
Access to XMLHttpRequest at 'https://jsonbox.io/box_xxx?query_key=asd&query_value=qwe' from origin 'http://localhost:3000' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.
This is wonderful,if has GUI
I need to monitor box usage.
Hey Vasanth,
Great work with jsonbox. Would you be open to the idea of having a config variable that allows for whitelisting an array of IP addresses for accessing self hosted jsonbox?
This would allow for the service to reside on a public IP address but be accessible only via VMs like on on DigitalOcean, or my local IP address etc. Would also make for quick way to spin up a private DB in the cloud where no stranger can simply create a record on my jsonbox.
Happy to raise a PR for this one if you'd like.
Thank you for your free service
test.http
POST https://jsonbox.io/box_03b46fcdb8926a1b2eef HTTP/1.1
Content-Type: application/json
{"name": "Jon Snow", "age": 25}
Response
POST https://jsonbox.io/box_03b46fcdb8926a1b2eef HTTP/1.1
User-Agent: vscode-restclient
Content-Type: application/json
accept-encoding: gzip, deflate
content-length: 31
cookie: __cfduid=d488182a5636f2c08c910a10be241f6781603012854
{
"name": "Jon Snow",
"age": 25
}
HTTP/1.1 401 Unauthorized
Date: Sun, 18 Oct 2020 09:21:56 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 30
Connection: close
X-Powered-By: Express
Access-Control-Allow-Origin: *
Etag: W/"1e-NSvI/xx+ygCisT8KmSK3s9GLVMI"
Via: 1.1 vegur
CF-Cache-Status: DYNAMIC
cf-request-id: 05dc9d0c4b0000d39a46079000000001
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report?lkg-colo=12&lkg-time=1603012917"}],"group":"cf-nel","max_age":604800}
NEL: {"report_to":"cf-nel","max_age":604800}
Server: cloudflare
CF-RAY: 5e413126da9fd39a-LAX
{
"message": "Invalid API_KEY."
}
A 512MB quota is very reasonable for this service.
When I POST to my box, I receive this:
> curl -X POST "https://jsonbox.io/box_cb9c0a7cb8c971182f81" -H 'content-type: application/json' -d '{"id": 0, "text": "do-me", "assigned": false, "done": false}'
{"message":"you are over your space quota, using 516 MB of 512 MB"}
My box is at https://jsonbox.io/box_cb9c0a7cb8c971182f81 (it's not sensitive content to me). Its size:
> curl https://jsonbox.io/box_cb9c0a7cb8c971182f81 | wc --bytes
1444
1.4MB, not 516MB.
This is a great service, and I like it. This issue isn't holding anything up, just curious why this is happening. Poked around a bit and couldn't see anything obvious.
Is there any limits to the number of records stored ?
Let's assume I stored 100 or 300 records for example, would that work?
Hi, first of all, thanks for this amazing project! I've been using for a while then I decided to publish a Java wrapper version I've been using since there's no JVM wrapper yet.
The source-code and documentation is already done.
However, unhappily Java world isn't that easy to publish a library ๐ , we have to open and ticket to request, so I did: https://issues.sonatype.org/browse/OSSRH-52070
But they are requesting to publish a text file in your domain to use the "jsonbox" name on the library, such as https://jsonbox.io/javalib.txt with the ticket ID to be able to use that name
Add a TXT record to your DNS referencing this JIRA ticket: OSSRH-52070
So I'd like to ask you that favor to be able to publish this library as "jsonbox" on Java, otherwise the name would be "com.github.leonardiwagner.jsonbox" ...yes, a horribly horrible name ๐
If you consider that when I got the approval to publish I update you to remove that .txt file from your domain, and I create the pull request to add this library in README.md
thanks! ๐
as the title said, I'm wondering how long will the data be keeped? 1 month? forever?
and also, any other limitations that not mentioned on the README?
The current number of records in Jsonbox is 218058
and its keep growing. I know this day will come that we might need to cut down the records. So I have plan for that and I need suggestions from people to find the best possible way to cut down the number of records, so that people can keep use it for free.
My suggestion:
Introduce lastAccessedDate
for boxes (could be GET, POST or PUT). So when this date is more than 6 months (could be 3 months also) we delete them.
A local version would remove a lot of demo effects in hackathons when the wifi is laggy, and would also be faster due to no network latencies.
The easy & dirty way would be to create a single container that runs both a mongo server and the node process, but this would make the container very heavy and problems hard to debug.
An alternative route would be to abstract the db layer and allow using something like sqlite3.
And yet another option would be to just provide an in-memory db option for local use. To make this a bit more usable, one might consider adding super simple to understand persistence by making it possible to enable a stdout output in the style of a log that can be piped back to a fresh server to restore the old state.
Example after providing a simple Dockerfile in the repo:
docker build -t jsonbox .
docker run --rm -i -p 80:80 jsonbox >> log.dump
[ do stuff and exit container ]
cat log.dump | docker run --rm -i -p 80:80 jsonbox >> log.dump
The log could look something like this, just outputting a line for each operation:
PUSH box_id/category/xxx { "i was": "inserted" }
PUT box_id/category/xxx { "i was": "updated" }
DELETE box_id/category/xxx
I personally would prefers the last one as it is by far the simplest to operate and allows storing different states of the db simply by copying the file, but understand if you don't want to add code for doing in-memory db alternative with custom query implementations.
Let me know if any pull requests would be welcome, especially for the last option :)
Would be great if there was a way to secure my own data/box.
I saw in the source code db.json
had something like that:
_createdBy
and _updatedBy
but nothing backs it up in the logic.
Is it possible to query using _createdOn:<=
or _createdOn:>=
?
I'm able to update a record using curl -X PUT
with a 34KB JSON file, but as soon as the file hits 35KB, I see: {"message":"JSON body is too large. Should be less than 50KB"}
. Am I doing something wrong, or is this a bug?
Thanks in advance!
You are using the README as documentation, so I think that translations could be very helpful. What do you think about it?
true
false
null
"true"
"false"
"null"
These are all valid JSON values, but can't all be queried at present. Right now, true
and false
are being cast to their primitive types when supplied as string queries (and null
is not):
q=booleanField1:true,booleanField2:false
Lines 84 to 85 in 6781bd2
I think it might be better to treat these non-string primitive values the way that numbers are being treated, but as special cases:
q=booleanField1:=true,booleanField2:=false,fieldWithNoValue:=null
Line 71 in 6781bd2
I haven't worked with mongo in a long time, so I'm not sure about the exact syntax, but something like
else if (value.startsWith('=')) {
if (val === 'true') query['data.' + key] = true;
else if (val === 'false') query['data.' + key] = false;
else if (val === 'null') query['data.' + key] = null;
else query['data.' + key] = +val;
}
and then you could just remove the code from lines 84โ85 above.
Is there a way to find records contains a comma in string field?
Steps to reproduce.
curl -X POST 'https://jsonbox.io/kuy_326c183fded9e7bc28a2xx' \
-H 'content-type: application/json' \
-d '{"name": "Hello, Jsonbox"}'
curl -X GET 'https://jsonbox.io/kuy_326c183fded9e7bc28a2xx?q=name:Hello,%20Jsonbox'
curl -X GET 'https://jsonbox.io/kuy_326c183fded9e7bc28a2xx?q=name:Hello%2C%20Jsonbox'
Both GET operations return Cannot read property 'startsWith' of undefined
error message.
It would be nice to have wrapper for C#
dot net core is preferred
i'm using my local database i want to extends the records
thank you..
This is an idea to improve the messaging/documentation.
The description of "private" boxes might be misleading to some because they can be read without the API key. "Protected" might be a better description because only mutation actions (create/update/delete) require an API key.
Hello guys,
I am experiencing a weird behavior. Whenever I am putting more than 20 records into a box the oldest one is removed so that the max records is never above 20 entries. Has somebody the same problem?
Chris
Similar to 6a96ce7 for size limit.
It could be interesting to add a hard cap of maximum records inserted, also letting users specify a cap (if lower than the max setting).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.