FTPS stops working with ZitaFTP Server from version 5.4 about amissl

[Futaura]

Does ZitaFTP have anything to do with setting the cipher suites up directly or indirectly? If you are relying solely on libcurl for this, are you able to enable debug logging for libcurl?

Obviously, it would seem related to something that changed in OpenSSL 3.0.6 (I see from the changelog that there were a bunch of changes made to fix custom ciphers).

[ksdhans]

libCURL is not involved on the FTP server side of things (libcurl is for making client requests).

Yes, ZitaFTP seets up cipher suites, calling the following:
SSL_CTX_set_min_proto_version()
SSL_CTX_set_cipher_list()
SSL_CTX_set_ciphersuites()

The chosen cipher suites are the ones that Mozilla recommends.

This does give me a deja vu feeling. Early in ZitaFTP development, a baserel problem in AmiSSL v4.3 caused TLS 1.3 & elliptical ciphers to fail (#35).

[Futaura]

Good, at least I haven't got to wade through the curl sources. IBrowse uses these three functions too. I'll take a look.

Presumably, you're using the "Intermediate" set of ciphers (TLS 1.2/1.3)?

[ksdhans]

The default is the "intermediate" set of ciphers, although users can change it if they want to (which they usually don't).

[Futaura]

Ok, so I agree that this does look a lot like #35 and I can confirm the #35 tests fail with 5.5, but work with 5.3. However, the code that was fixed in #35 didn't change between 5.3 and 5.4/5.5, and I've confirmed the previous workaround is still in place. Also, your tests work with OpenSSL 3.0.7 on Linux. So, the problem must be elsewhere - I have some more hunting to do.

[Futaura]

I'm still working on this, but it is going to take a while longer. It is actually caused by the same bit of code as in #35, but now the workaround that I added (array moved to .data) is giving the same end result as the original code (array in .rodata).

I have made a different workaround which ensures the array is put on the stack instead, which uses the correct baserel references, but I need to first find out why this is failing in the first place in case it is a bigger issue. Given the OpenSSL code in this area did not change, I fear it may actually be related to my fix for #49. I think I've going to have to build a debug elf.library to figure out what is going on in elf.library/CopyDataSegment(), as the source code shows that it ignores certain errors and it may be that something is going wrong with the relocs.

[ksdhans]

Thanks for the update. I hope you can find a permanent fix.

[Futaura]

Ok, so I'm going to spew out all the technical info here, mainly so that it is here for future reference (otherwise I'll forget it all).

It is definitely related to the #49 changes. Although elf.library/CopyDataSegment() is not returning a failure, it is actually silently failing to apply all relocations in the copied data segment. This is because the relocs are never fully loaded into memory in the elf handle, so it always tries to read them from the compiled library on disk, but my change in #49 closes the file handle in the elf handle, causing the reads to fail. This may sound catastrophic, but fortunately all relocs in the .data section are non-relative, so the original .data section has the relocs applied during LoadSeg() and those get copied along with the data segment. The only issue is with relocs in .data which point to symbols inside .data, of which there are 5 such references. 3 of those references come from the alltabs[] in ssl3_get_cipher_by_std_name() which is the code causing this issue (and in #35).

Unfortunately, this all means it is looking like I will have to revert my fix from #49, which means a read lock will again be maintained on the libraries until they are expunged. Without some major changes in elf.library, this is always going to be required, due to the relocs not being stored completely in memory and only in a fixed size temporary buffer which is refilled as required (of course, for non-baserel applications, they are not required after LoadSeg()). One potentional solution is to open and close the file handle each time the library is opened - however, if the user should update AmiSSL while it is running, this will cause a crash the next time the library is opened as it will load the relocs from the new library and not the old library running in memory. However, I still desperately want to avoid the read locks, so if elf.library can already provide enough information, I may try to write my own reloc code instead, storing the required relocs in memory (if it turns out not to be possible, I may try adding some new functions to elf.library to help in handling this).

Also, to fully document the original issue from #35 (although I had left comments, I had forgotten the detail), the cipher structures in openssl/ssl/s3_lib.c are I think read only, except that the arrays get sorted with qsort (hence why they are not const and not in .rodata). It is only the fact the arrays need to be sorted which triggers the issue, as they only get sorted in the copied .data sections and not in the original, otherwise it wouldn't matter that alltabs[] in ssl3_get_cipher_by_std_name() got placed in .rodata and not in .data. Therefore alltabs[] must be placed in .data and not .rodata as the pointers there need to be base-relative (relocs to .data in .rodata will only ever point to the original .data, not the copy). I still think this is compiler bug, as it should know not to place such references in .rodata, but I will try to see if there any command-line options for GCC that will prevent this. I do also need to check again the other existing .data relocs in .rodata. It might be avoided using .sdata and the latest GCC, but this is not currently possible due to -mcheck68kfuncptr option which was removed after GCC 4.2.4.

[ksdhans]

That's a nasty bug in the elf.library. Have you sent a bug report to Hyperion yet?

[Futaura]

No, but then it could be rightfully argued that I shouldn't be causing the file handle to be closed before using elf.library/CopyDataSegment() which is a little bit of a hacky thing to do on my part, just to release the file lock. IIRC, this function was introduced for AmiSSL originally, so probably nobody else uses it (except IBrowse). I'm quite happy for it to remain as-is for now, otherwise there will be no way to use CopyDataSegment() at all without maintaining the file lock as in older versions. Besides, probably nobody will be too keen to fix it - I can certainly fix it myself when the time comes.

I checked last night and writing reloc code in AmiSSL is not really feasible because it involves symbol lookups too, and really that kind of access is not given to applications. I'm thinking instead to either introduce new mechanisms in elf.library and/or heavily revising CopyDataSegment() so that perhaps even the required relocs are automatically cached in memory when ElfLoadSeg() detects baserel relocs in .text.

In the meantime, in AmiSSL I am going to use the new workaround above of ensuring the problem array is placed on the stack and not in .rodata or .data. Then I'll hunt down the other two problem relocs and add a workaround for them too, if required. If there were lots of relocs, I wouldn't want to bother, but it shouldn't be too hard to manage 5. I'll also write a script to check for any new problem .data relocs, at build time, that may appear in future OpenSSL versions.

[Futaura]

As you may have seen on the dev list, I've now rewritten CopyDataSegment(), which no longer requires any disk access. I hope to commit the changes and release a new beta by the end of the week.

I keep changing my mind on what to do with AmiSSL. I think it may just be safer to revert to the v5.3 behaviour and keep the file lock open, as I can't be sure that my trick in v5.4/5.5 doesn't cause any other side effects or whether older versions of elf.library behave worse. For v5.6, it would only open the elf handle with no i/o if using elf.library 53.35 or higher. Would this cause you any trouble the ZitaFTP installer, etc?

[ksdhans]

The ZitaFTP installer shuts down the server before updating, so the old behaviour should work well enough. Of course, it'll still need a reboot if something else is using AmiSSL, but that's something we're going to have to live with for now.

[Futaura]

Fixed in AmiSSL 5.6 (or elf.library 53.35).