Comments (9)
Now the latest release v0.27.1 follows the new release convention. I decided to compress the deliverables because amalgamated sqleet.c
is quite big (7.6MB) and including README.md
& shell.c
in release archives is nice. I also deleted the two oldest releases (v0.1.0 & v0.1.1) that did not use the current version numbering scheme, which hopefully avoids confusion.
Older releases (<0.27.1
) can remain in the old release format.
Closing the issue. Reopen if you find mistakes or shortcomings in the new release approach.
from sqleet.
Yes, there was an issue with sqlite-v0.26.0
tag. A quick workaround for your use-case is to checkout this way: git checkout -b asdf sqleet-v0.26.0~1
(all release branches & tags point to an amalgamation commit that directly forks from master branch). This works for now, but in the future we'd like to have a better system, so leave this issue open until that happens.
Sorry in advance for the wall of text. And big thanks for your patience with the project so far, especially since the user experience has not been the smoothest possible. Feedback about how people use sqleet is valuable in order to improve the library.
The core issue is that branches, tags & releases are a bit abused in the repo for reason(s) not immediately obvious. The main requirement is that we want to offer old sqleet release amalgamations targeting specific SQLite3 versions in case some users are locked into a specific SQLite3 version. Currently the old releases are unmaintained, i.e., bug fixes are not backported to them. This is clearly not ideal but I do not see an easy way to address this without substantial amount of boring work needed to support (already) ten releases. *
Other requirements are imposed by Github's clunky release feature. Using a tag name v0.27.1
yields a source package named v0.27.1.zip
, thereby requiring the sqleet-
prefix to get sane file names (like sqleet-v0.27.1.zip
). This on the other hand has the drawback that the source directory contained in the package is named sqleet-sqleet-v0.27.1/
which is maybe even more stupid. In other words, one simply cannot win (although it is possible that I'm just a bit OCD to care about these details at all).
Anyway, I have been recently planning to resolve the current releases/branches/tags mess as follows:
- Get rid of the release branches and make release tags point to a location in the master branch instead of an amalgamation commit (this is the expected behavior by you if I understood correctly).
- In the unlikely case that an old sqleet release needs support, fork a release branch from the master (specifically, from the location pointed by the existing release tag) and create a bug fix release.
- Offer release amalgamations (
sqleet.c
,sqleet.h
) as uploaded "release binaries" because Github's source code packages are fundamentally broken unless I'm missing some detail of how releases work.- Consequently, source code packages can be fully ignored so the file names become unimportant. The packages would contain unamalgamated sqleet sources matching the amalgated files.
- Figure out what to do with releases published to date. Migrating them into this new system is perhaps feasible and better than leaving them as is.
This is what I would do now knowing the intricacies of Github more thoroughly. Any comments or ideas before I rush to implement this?
(*) As a side note:
If someone were to request support for an old sqleet release (via the Github issue tracker), then I could make it happen. However, it is still quite straightforward to take a bug-fixed newer version of sqleet and compile it against old SQLite3 (some trivial changes to the code may be needed; for example, sqlite3RunVacuum()
interface changed in 3.27.0 which broke backward compatibility with previous SQLite3 versions -- this could definitely be improved by #ifdef
fing against SQLite3 version numbers, but I'm unsure if it is worth the effort).
from sqleet.
Prepare your own SQLite3 amalgamation and just replace sqlite3.c
& sqlite3.h
in sqleet's master. Compile and you should be good to go. If you want to produce sqleet release style drop-in replacements of your custom amalgamation version, then commit the changed SQLite3 files to master branch and run ./script/release.sh
in a Linux environment. It should look like this:
[sqleet]$ ls
config.c README.md script sqleet.c sqlite3.h
crypto.c rekeyvacuum.c shell.c sqlite3.c UNLICENSE
[sqleet]$ ./script/release.sh
[+] SQLite version 3.26.0
[+] Generating rekeyvacuum.c
[+] Generating sqleet.c amalgamation
[+] Updating shell.c #include "sqlite3.h" -> "sqleet.h"
[+] Moving files around a bit
[+] Success!
git checkout -b v0.26.0
git commit -m "Release v0.26.0"
git push -u origin v0.26.0
[sqleet]% ls
README.md shell.c sqleet.c sqleet.h UNLICENSE
Now sqleet.c
and sqleet.h
contain your version of SQLite3 amalgamation with sqleet encryption support. Note that the printed git commands are not executed. They are there to simplify the release procedure for sqleet maintainers (i.e., a fancy way to say 'me').
If you cannot run the release.sh
script in your environment for whatever reason, you can still manually make the sqleet-enabled amalgamation quite easily: Replace the few #include
s in the beginning of sqleet.c
with the contents of the included files and you're done. The only nasty part is updating rekeyvacuum.c
to match the SQLite3 version used, but it is not mandatory most of the time (the release script executes ./script/rekeyvacuum.sh
which tries to automate this process).
Also, if you are unlucky, the result may not compile straight out of the box if the used SQLite3 version differs enough. Fortunately, the required changes have been so far trivial across several years in the past (e.g., slightly adjust calls to SQLite3 functions which have been renamed or updated to take one additional parameter etc).
I agree that the procedure could be easier as it is in the official SEE extension case. However, I don't like the idea of making sqleet concatenable like SEE, because then we would lose the control of SQLite3 compile-time configuration (config.c), which is crucial for security reasons. Maybe writing a separate script or improving release.sh
might be acceptable route forward. What do you think?
from sqleet.
Prepare your own SQLite3 amalgamation and just replace sqlite3.c & sqlite3.h in sqleet's master. Compile and you should be good to go.
I originally created this issue with the goal of creating drop-in replacement amalgamations, but on second thought, this method actually does satisfy my needs.
I'll be using the same versions of sqlite as sqleet (no mis-matched versions)—the only difference is that my amalgamation will be created using various non-default configuration (e.g., OMIT_*
flags, which must be present during sh configure
within the sqlite source repository).
As far as I can tell, none of those OMIT_*
flags affect sqlite3RunVacuum()
until the C preprocessor actually runs. So I should just be able to use the existing rekeyvacuum.c
file from the branch I'm using (currently sqleet-v0.26.0
), without needing to run ./script/rekeyvacuum.sh
.
While it turns out my needs are already met, I would recommend simplifying release.sh
(or providing an alternative to it) that exhibits more friendly behaviors:
- It shouldn't require that the
master
branch is checked out, or that the working directory is clean. People would more likely want to check out a reliable tag (e.g.,sqleet-v0.26.0
) rather than the ever-changing master branch. And in my case, the working directory won't be clean if I move in my ownsqlite3.c
file. - It should be more portable. For example,
sed
on Mac OSX doesn't have the-r
flag. That's the first thing I noticed, but I bet there are other non-portable elements in that script as well. Of course, I'm biased because I develop on Mac OSX.
In any case, thanks for making this great library.
from sqleet.
As far as I can tell, none of those
OMIT_*
flags affectsqlite3RunVacuum()
until the C preprocessor actually runs. So I should just be able to use the existing rekeyvacuum.c file from the branch I'm using (currentlysqleet-v0.26.0
), without needing to run./script/rekeyvacuum.sh
.
That sounds OK to me.
While it turns out my needs are already met, I would recommend simplifying release.sh (or providing an alternative to it) that exhibits more friendly behaviors: ...
Commit 0dc5a18 added script/amalgamate.sh
which performs amalgamation of sqleet master branch sources (instructions added to README.md
as well). release.sh
still has the annoying master branch & cleanliness check, but that is probably no longer an issue because the amalgamation procedure can be executed directly via the new script.
I also removed the -r
flag from sed invocations. Unfortunately I do not have access to Mac OS X so I cannot solve remaining portability problems (if there's any). At least bash, busybox sh, fish & zsh successfully produce identical amalgamation using the script/amalgamate.sh
script, which is promising. If you want to test on Mac OS X using current master:
[sqleet]$ git rev-parse HEAD
0dc5a18dbf93972316ea21315767323b2509befb
[sqleet]$ ./script/amalgamate.sh <sqleet.c | md5sum -
4ea5dedea10d9d538d2f06097edb45cc -
from sqleet.
This is great, thanks!
from sqleet.
@resilar I just found out (via #20) that release branches and tags are supposed to be post-amalgamation. I started on sqleet-v0.26.0
, and saw that it wasn't amalgamated, and assumed that was an intentional pattern, but it turns out I was mistaken.
Since I need to compile against my own version of sqlite3.c
, I have to do the amalgamation myself—I cannot use those amalgamated branches. However, I would like to use stable released versions of sqleet, rather than just using master
which is inherently unstable. So it turns out this issue is actually unsolved.
from sqleet.
A quick workaround for your use-case is to checkout this way:
git checkout -b asdf sqleet-v0.26.0~1
(all release branches & tags point to an amalgamation commit that directly forks from master branch)
This does solve my problem for now, thanks.
Sorry in advance for the wall of text. And big thanks for your patience with the project so far, especially since the user experience has not been the smoothest possible.
Thank you for making this great library and putting it in the public domain!
Get rid of the release branches and make release tags point to a location in the master branch instead of an amalgamation commit
This would be ideal. From my perspective, release tags/branches should always be on a regular git timeline, and special pre-built deliverables should be delivered separately.
In the unlikely case that an old sqleet release needs support, fork a release branch from the master (specifically, from the location pointed by the existing release tag) and create a bug fix release
In this case, what would the bugfix release be named/tagged as? Given how the semver numbers are used to match the sqlite3 version, I assume the sqleet patch would have to be indicated by some other naming convention, perhaps appended to the sqlite3 version number?
- Figure out what to do with releases published to date. Migrating them into this new system is perhaps feasible and better than leaving them as is
Doing this on a per-request basis is probably good enough. I can't imagine that too many people would have such a rigid need for backwards compatibility, given that this project is relatively young.
from sqleet.
Thanks for your input.
In this case, what would the bugfix release be named/tagged as? Given how the semver numbers are used to match the sqlite3 version, I assume the sqleet patch would have to be indicated by some other naming convention, perhaps appended to the sqlite3 version number?
Yes, naming bug fix tags can be tricky because of the weird way sqleet versions are numbered to match SQLite3 versions (although I don't see an easy way to fix the tag naming issue by changing the version numbering convention). AFAIK semver allows appending a suffix to version number (e.g., 0.27.1-fix0
) so maybe that could be used as you suggested. Of course simply recreating the old release tags without changing names is also an option, but that can cause major confusion in the long run if the same version number can correspond to multiple commits. Thus, appending suffixes is probably the way to go.
I can't imagine that too many people would have such a rigid need for backwards compatibility, given that this project is relatively young.
Yes. Given the age of the project, I'm considering updating existing releases to the new system. It should not cause significant breakage for users, I think.
from sqleet.
Related Issues (20)
- Problem with memory databases and encryption HOT 13
- Error compiling in Xcode - Clang HOT 2
- sqleet v1 cryptosystem HOT 5
- PRAGMA for header, salt, kdf rounds and skip
- Sqleet Shell's URI Interface miss(behavior) across different platforms & compilers
- SQLite 3.32.3 build issues HOT 3
- What are the plans for SQLeet? HOT 17
- Any way to compile on MacOS? HOT 4
- Database size limit
- Fatal signal 11 after set rekey for unencrypted database HOT 1
- Question: Anyway to open encrypted DB with DB Browser SQLite (SQLCipher)? HOT 4
- Anyway to fix broken encrypted database? HOT 2
- How to use SQLeet encryption for Room?
- compilation issue HOT 1
- Question: Is there a way to check if a database has been encrypted in sqleet? HOT 4
- Bugs from SQLite 3.31.1 in sqleet
- compile error libsqleet
- [Fixed] encryption on Android not working
- How to support such a statement "ATTACH DATABASE x AS y KEY z"? HOT 1
- Cannot open SQLCiper database
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sqleet.