laplacekorea / ws4sqlite

Query sqlite via http

🌱 ws4sqlite

ws4sqlite is a server application that, applied to one or more SQLite files, allows to perform SQL queries and statements on them via REST (or better, JSON over HTTP).

Possible use cases are the ones where remote access to a sqlite db is useful/needed, for example a data layer for a remote application, possibly serverless or even called from a web page (after security considerations of course).

Client libraries are available, that will abstract the "raw" JSON-based communication. See here for Java/JVM, here for Go(lang); others will follow.

As a quick example, after launching

ws4sqlite --db mydatabase.db

It's possible to make a POST call to http://localhost:12321/mydatabase, e.g. with the following body:

{
    "transaction": [
        {
            "statement": "INSERT INTO TEST_TABLE (ID, VAL, VAL2) VALUES (:id, :val, :val2)",
            "values": { "id": 1, "val": "hello", "val2": null }
        },
        {
            "query": "SELECT * FROM TEST_TABLE"
        }
    ]
}

Obtaining an answer of

{
    "results": [
        {
            "success": true,
            "rowsUpdated": 1
        },
        {
            "success": true,
            "resultSet": [
                { "ID": 1, "VAL": "hello", "VAL2": null }
            ]
        }
    ]
}

Features

Docs and a Tutorial.

• A single executable file (written in Go);
• HTTP/JSON access, with client libraries for convenience;
• Directly call ws4sqlite on a database (as above), many options available using a YAML companion file;
• [In-memory DBs] are supported (https://germ.gitbook.io/ws4sqlite/documentation/configuration-file#path);
• Serving of multiple databases in the same server instance;
• Batching of multiple value sets for a single statement;
• All queries of a call are executed in a transaction;
• For each query/statement, specify if a failure should rollback the whole transaction, or the failure is limited to that query;
• "Stored Statements": define SQL in the server, and call it from the client;
• CORS mode, configurable per-db;
• Maintenance scheduling (VACUUM and backups), also configurable per-db;
• Builtin encryption of fields, given a symmetric key;
• Provide initialization statements to execute when a DB is created;
• WAL mode enabled by default, can be disabled;
• Quite fast!
• Compact codebase (~900 lines of code);
• Comprehensive test suite (make do-test);
• Docker images, both for amd64 and arm32.

Security Features

• Authentication can be configured
  • on the client, either using HTTP Basic Authentication or specifying the credentials in the request;
  • on the server, either by specifying credentials (also with hashed passwords) or providing a query to look them up in the db itself;
• A database can be opened in read-only mode (only queries will be allowed);
• It's possible to enforce using only stored statements, to avoid some forms of SQL injection and receiving SQL from the client altogether;
• CORS Allowed Origin can be configured and enforced;
• It's possible to bind to a network interface, to limit access.

Design Choices

Some design choices:

• Very thin layer over SQLite. Errors and type translation, for example, are those provided by the SQLite driver;
• Doesn't include HTTPS, as this can be done easily (and much more securely) with a reverse proxy;
• Doesn't support SQLite extensions, to improve portability.

Credits

Many thanks and all the credits to these awesome projects:

• lnquy's cron (MIT License);
• robfig's cron (MIT License);
• gofiber's fiber (MIT License);
• mitchellh's go-homedir (MIT License);
• mattn's go-sqlite3 (MIT License);
• wI2L's jettison (MIT License)
• DataDog's zstd (Simplified BSD license);
• and of course, Google Go, VS Code and CodeServer!