Bundle SQLite3 dynamic library about sqlite3

[DjDeveloperr]

Bundling the SQLite library into this module would require bundling for all three platforms, and two different architectures for macOS, which would be a big size increase when downloading and caching module, also there is no way to bundle a native binary into JS module. There are some hacky approaches like base64 encoding and writing as a string in JS file, but it really only has downsides IMO. Only viable option seems dynamically downloading at runtime and caching the binary.

[DjDeveloperr]

Now this library downloads and uses its own prebuilt version of SQLite3 which ensures safety and consistency across platforms. I'll be closing this issue as fixed

[dyedgreen]

There is also https://deno.land/x/sqlite which bundles a sqlite WASM build

[DjDeveloperr]

There is also https://deno.land/x/sqlite which bundles a sqlite WASM build

Yep, sqlite (wasm) is able to do that by encoding wasm binary in base64 and writing to a JS file which can be simply imported and automatically cached by Deno. That approach is viable only for wasm since it has same binary for all OS and arch.

Downloading and caching native libraries at runtime is something we can consider. Ideally, sqlite should be built in CI and published along with release and that release URL to be used by code to download.

[waynebloss]

Hello - if I wanted to make my own solution for this, where would I deploy the [lib]sqlite3.dll/so/dynlib file on the end-users system? Right next to the binary for my app?

Also, would that same libsqlite3 file be needed during Deno compilation? In other words, do I need to supply the DENO_SQLITE_PATH during build-time (compiling) or at runtime? I am not familiar with how FFI works, but I am guessing that the libsqlite3 linked library would not be necessary when building my app's binary, with Deno.

If I put libsqlite3 right next to my app's binary, will it automatically get picked up from there?

Looking at the code now, I see that you call Deno.dlopen and I would guess that just works at runtime exclusively. However, I did not find out much about the search path that it uses.

I'd also like to not have to build sqlite into my CI pipeline if I didn't have to - is there a well known set of sqlite3 builds that I can draw from?

(I can probably figure these out soon; I'm still debating if I want to try and use Deno for this app that I have in mind.)

Thanks!

[DjDeveloperr]

where would I deploy the [lib]sqlite3.dll/so/dynlib file on the end-users system? Right next to the binary for my app?

Yeah, CWD is one place we look for in first

but I am guessing that the libsqlite3 linked library would not be necessary when building my app's binary, with Deno.

That's right, we do dynamic linking

Actually this might be something desirable now. Recently found out that SQLite3 compiled with certain flags can increase the performance of the library and provide more consistent results across platforms. But we sure can't bundle, just shipping our own binary would be the goal.

[DjDeveloperr]

As an update, we now use custom built SQLite3 libraries on macOS and Linux since it has been observed to improve performance. Same could be done for Windows but there were not any performance gains so there we still use system library which is present in most cases.