SQLite dependency causes version problems about mimekit

[jstedfast]

#48 and #52

[jstedfast]

There is a MimeKitLite solution that you can use that doesn't depend on SQLite. I don't really want to maintain any more combinations if I can avoid it.

FWIW, MimeKit.dll doesn't depend on any specific version of SQLite.dll, you could just overwrite it with your own. I only include it for convenience.

[jstedfast]

Does this address the problem you are facing? Or am I misunderstanding the problem?

[fubar-coder]

I guess that my problem can be split into two parts:

1. The SQLite dependency is annoying and it doesn't allow an alternate way to provide certificates
2. It would be nice to have two NuGet packages. MimeKit (without the SQLite dependency) and some kind of interface implementation to use SQLite as backend for the certificate stuff

This change would allow me to use all features of MimeKit and provide a custom certificate store (if needed). This would also mean, that there is no need for a separate MimeKit Lite NuGet package.

Another benefit would be that the users of this library won't be forced to recompile the packages themselves.

To be honest: I really, really like this library, but the SQLite dependency is a no-go (because we're using System.Data.SQLite.MSIL and switch between the x86/x64 version depending on the environment the library is used with) and we're forced to stick with MimeKit v0.15.0.

[jstedfast]

Thanks for the explanation, I understand the problem much better now.

[mwpowellhtx]

I tend to agree with this concern: remove the Sqlite dependency. That's not a Pop3 (or an Imap, or any other client) concern, but rather a mail reader concern, if at all. Additionally, I don't need to get into 32- or 64-bit concerns right now, either.

[jstedfast]

Is there a NuGet package for the System.Data.SQLite.MSIL that you are using? Maybe this one? http://www.nuget.org/packages/System.Data.SQLite.Core/

Perhaps an easier solution is to simply have the .Net40 project depend on that instead of Mono.Data.Sqlite.

I'd really prefer not to split out a single class into its own assembly. Plus, it'd require users to implement and register their own subclass of SecureMimeContext, which I'd really rather avoid forcing people to do, because I want it to be usable even by novices.

I'm open to proposals, but I haven't figured out a clean way of splitting things out that I'm happy with.

[fubar-coder]

It's https://www.nuget.org/packages/System.Data.SQLite.MSIL/ .

I always have a bad feeling when it comes to those kind of dependencies. I still believe that some kind of API for the certificate validation/management would be the cleanest solution. What about using a method like the one used by the "NHibernate.NLog" NuGet package? It just adds a source file to your project which just implements the required interface. Nice, simple, no additional assembly.

[jstedfast]

I don't think I quite understand how that would solve things... do you think you could send me a patch to illustrate what you envision?

Wouldn't that file still depend on Mono.Data.Sqlite? Unfortunately, the API for Mono.Data.Sqlite and System.Data.Sqlite are not the same.

FWIW, I chose Mono.Data.Sqlite because it is available on all platforms: Windows, Linux, Mac, iOS, Android, etc. System.Data.Sqlite is unfortunately only available for Windows.

[kivikakk]

I'm finding this problematic for the same reasons; want to use the nice MIME parser, but can't have a conflict with System.Data.SQLite.Core (in my case).

What's the concern about a separate assembly for the crypto stuff? It feels like that level of pluggability is a good thing, not a bad one.

[jstedfast]

Pluggability is fine, I just was hoping to find a way of doing it that would minimize the amount of code that someone using MimeKit would have to write on their own.

[fogzot]

I think that a good way to provide plugability, remove the SQLite dependency and be gentle to novice coders would be to:

1. Introduce an appropriate API (ICertStore?)
2. If the user doesn't configure a specific ICertStore, use the SQLite one but make it load the DLL using reflection and then use just the System.Data (and System.Data.Common) interfaces to avoid hard-coding the DLL dependency.
3. If an appropriate SQLite DLL can't be loaded fall back to a filesystem-based store or even a "null" store.

I can even give a try at coding this if you like the idea.

[jstedfast]

I envision something like this:

IDbConnection sqlite = CreateSqliteConnection ();
IX509CertificateDatabase dbase = new X509CertificateDatabase (sqlite);
var ctx = new DefaultSecureMimeContext (dbase);

IX509CertificateDatabase already exists and is already generic enough to use (but needs to be made public, along with all of the X509*Record classes).

X509CertificateDatabase is also already implemented as an abstract class (again, needs to be made public) and does a lot of the work with no dependency on Mono.Data.Sqlite, it sticks to just System.Data interfaces.

SqliteCertificateDatabase subclasses X509CertificateDatabase and contains all of the bits I couldn't figure out how to abstract away the last time I started looking into this.

Basically, SqliteCertificateDatabase is mostly needed because of things like this:

command.Parameters.AddWithValue ("@ISSUERNAME", issuerName);

I think the problem was that IDbCommand.Parameters doesn't have an AddWithValue() or something like that, and I didn't know how to proceed, so I split it out temporarily into SqliteCertificateDatabase.

Also I wasn't sure if the SQL syntax I'm using is safe to use for all IDbConnection implementers.

[fogzot]

Thank you for the explanation/roadmap. I'll have a look and see if I can help.

[jstedfast]

Something I just learned about, which may be useful, is that Microsoft just released some new PCL binding to SQLite that works on Windows Phone (and Windows Store?) and Xamarin wants to bring it over to iOS, Android, Mac, and Linux so this may become an option as well.

I'm not sure if it implements the System.Data interfaces or not, though. From what I heard, it is a very low-level binding, so probably not.

[jstedfast]

@fogzot thanks for your patch to implement a lot of what I described above!

I've done a bit more work on top of your patch and things are moving along nicely. It is now possible to plug in your own certificate database - you can even re-use SqliteCertificateDatabase by providing your own IDbConnection.

Do you happen to know if the SQL syntax I've written is compatible with all SQL databases? If so, I might rename SqliteCertificateDatabase to SqlCertificateDatabase in order to make it more clear that it can be used with any SQL-based IDbConnection.

[jstedfast]

I think it's safe to call this feature request implemented now...

[CThuleHansen]

So just for a recap - is it possible to use both x64 and x86 now? and is it necessary to change anything?

[fogzot]

@CThuleHansen yes, but note that if you use DefaultSecureMimeContext Mono.Data.Sqlite is still loaded at runtime and unless you have the correct version somewhere you get an exception.

[fogzot]

@jstedfast at first sight the SQL seems fine to me. I'll test it on MySQL and PostgreSQL in the following days.

[jstedfast]

@fogzot awesome, thanks

@CThuleHansen Yea, you'll be able to use it on 32-bit and 64-bit now, but what you'll need to do is probably link your app with one of the System.Data.Sqlite packages that support both platforms and then register a subclass of DefaultSecureMimeContext that looks something like this:

using System.Data.SQLite;
using MimeKit.Cryptography;

class MySecureMimeContext : DefaultSecureMimeContext
{
    public MySecureMimeContext () : base (OpenDatabase ("C:\\wherever\\certdb.sqlite"))
    {
    }

    static IX509CertificateDatabase OpenDatabase (string fileName)
    {
        var builder = new SQLiteConnectionStringBuilder ();
        builder.DateTimeFormat = SQLiteDateFormats.Ticks;
        builder.DataSource = fileName;

         if (!File.Exists (fileName))
            SQLiteConnection.CreateFile (fileName);

        var sqlite = new SQLiteConnection (builder.ConnectionString);
        sqlite.Open ();

        return new SqliteCertificateDatabase (sqlite, "password");
    }
}

Then you can register your subclass as the default SecureMimeContext with the following snippet:

CryptographyContext.Register (typeof (MySecureMimeContext));

I should probably post this snippet in the README to illustrate how to do this since it'll probably be a common question.

If you have any more questions, let me know and I'll do my best to answer them.

[jstedfast]

This System.Data.Sqlite NuGet package is probably the one you want: https://www.nuget.org/packages/System.Data.SQLite/

[jstedfast]

I wonder if we can use reflection to check if System.Data.SQLite is available as well, to make this simpler.

Might also be able to make the MimeKit NuGet package depend on System.Data.SQLite?

[fogzot]

My idea was to implement different providers and fall back to the first one available. Adding System.Data.SQLite is easy but I don't know if I can test it. Is Linux or Windows with Xamarin Studio (no VisualStudio) enough?

[jstedfast]

I can probably test it on Windows since I have Windows and Visual Studio 2013 installed on my home desktop

[fogzot]

Looked at the SQL. Basically all the queries are OK but the DDL statements use some features supported only by SQLite (and partially by MySQL). For example AUTOINCREMENT is serial on PostgreSQL and IDENTITY (?!) on SQL Server. We have two possibilities here: a shallow hierarchy where each provider reimplements almost all methods or a deep one where an abstract SqlCertificateDatabase implements all the queries and leaves to concrete subclasses the DDL and connection creation. If you agree I'd like to implement the second one and provide an example NpgsqlCertificateDatabase (including tests).

EDIT: Should we open a "feature" issue where to track this? (Just because this bug is closed now.)

[jstedfast]

Yea, I think I like the second approach

[jstedfast]

...And yea, let's open another issue about this :)

[kae36]

Is there a link to the issue that was opened that could be added in the comments?