OS3/m68k issue: 'ssltest -test_cipherlist' fails about amissl

[Futaura]

Ouch - that's nasty. Obviously, when calling functions in this way from an application, there is no way to restore the data pointer. This could potentially have an impact on the OS4 version too. So, now we know another reason why OPENSSL_GLOBAL was set to AMISSL_COMMON_DATA! Of course, if the data is const, on OS4 it goes in .rodata anyway, so no problem. I presume that you tested if the __far keyword does anything on the m68k compiler when using baserel? And is there no way to force const data into .text? I fear there is not going to be any way around this without being able to conditionally force variable data references to be absolute, rather than baserel as can be done on OS4 and with SAS/C. That said, applications would generally be using the public function calls, rather than calling directly via those function points.

[jens-maus]

Yes, I tested that GCC indeed doesn't support the __far statement. We have, however, already contacted Gunther Nikl which is an expert in m68k GCC related things. I hope he will answer soon and provide a solution/patch for the GCC 2.95.3 which will allow to at least put const data into the .text section instead. Otherwise we will have to dig into the GCC sources ourself, I am afraid :(

Regarding AMISSL_COMMON_DATA, I still feel that we can stay away from that for a minute because most other global data is indeed defined const. But I think we should again carefully check if there is another global table not being const potentially causing trouble even on OS4. Perhaps the easiest way would be to dive through the .map file and see which openssl data structures are not in the .rodata section.

[tboeckel]

Olver is right, the functions from the returned structures are called without the necessary context for base relative addressing and hence the function access the wrong data. This is no problem for the OS4 build in case constant data in the .rodata section is accessed. But as soon as variable data in the .data section is accessed the OS4 build will cause the same kind of problems as the OS3 build.

Here are some static non-constant data definitions which are in fact constant:
ssl/s3_enc.c: ssl3_pad_1, ssl3_pad_2
ssl/t1_lib.c: nid_list, suiteb_sigalgs, tls12_md, tls12_sig, tls12_sigalgs
ssl/d1_both.c: bitmask_start_values, bitmask_end_values
ssl/d1_srtp.c: srtp_known_profiles
ssl/ssl_ciph.c: ssl_handshake_digest_flag, ssl_mac_pkey_id, ssl_handshake_digest_flag

I might be wrong with some of these variables. I just did a quick scan. It might be possible that some variable are indeed written to.

Just take a look at the generated .map file. There are lines like these:
.data 0x00000000002a0168 0x0 ./bin_os4/openssl/libssl.a(t1_clnt.o)
.data 0x00000000002a0168 0x1b4 ./bin_os4/openssl/libssl.a(t1_lib.o)
0x00000000002a01b0 TLSv1_1_enc_data
0x00000000002a01f8 TLSv1_enc_data
0x00000000002a0168 TLSv1_2_enc_data

This means that there are no global variables in t1_clnt.c, but global variables of 0x1b4 bytes in t1_lib.c, namely TLSv1_1_enc_data, TLSv1_enc_data and TLSv1_2_enc_data. If the variable names are missing the variables are declared as static.

Perhaps those non-const *_enc_data variables can be declared as const, but very probably these will result in lots of warnings. Furthermore I am absolutely not sure whether these structures are really constant or not. All this is just a very quick and very wild guess.

[jens-maus]

Looking at the latest master branch of the OpenSSL project it seems the OpenSSL developers have finally identified these structures to be constants as well:

https://github.com/openssl/openssl/blob/master/ssl/t1_lib.c

The master branch is actually based on the upcoming openssl 1.1.x version which is already available as a pre-release. So either we simply define the structures const ourself or I can upgrade our openssl sources to the latest 1.1.x sources which are, however, not flagged stable yet.

[tboeckel]

It is definitely possible to explictly place variables in a specifc section by adding something like __attribute__((section (".text"))) to the definition. Unfortunately this still doesn't work as it triggers an internal linker error due to the still base relative access to the variable. So it seems it is not just the linker that needs to be modified, but the compiler as well to not generate base relative accesses for constant global variables.

This is definitely a task for Gunther Nikl. I have no clue where to look in gcc's source code, nor do I know what to change at all.

[jens-maus]

Here a small example how to demonstrate that the current GCC 2.95.3 is putting baserel data always in the .data section even if it is defined as 'const':

Example source:

#include <stdio.h>

static const int str[1000] = {0};

int main(void)
{
    printf("arr is %d\n", str[0]);
    return 0;
}

(1) compile without -fbaserel32 and show section size:

$ m68k-amigaos-gcc -m68020-60 sectiontest.c
$ m68k-amigaos-size a.out
   text    data     bss     dec     hex filename
  19588     528     440   20556    504c a.out

(2) compile with -fbaserel32 and show section size:

$ m68k-amigaos-gcc -m68020-60 -fbaserel32 sectiontest.c
$ m68k-amigaos-size a.out
   text    data     bss     dec     hex filename
  15076    5280       0   20356    4f84 a.out

Please note the difference in the data section size between (1) and (2). Thus, turning on baserel with the m68k-amigaos-gcc 2.95.3 is putting const data into the data section rather than in the .text section. Using the same example for the ppc-amigaos-gcc demonstrates that there using -mbaserel correctly puts const data not into the data section.

[jens-maus]

Please note that the recent gcc 2.95.3 version available at https://github.com/cahirwpz/amigaos-cross-toolchain has been modified to put const data into the .text section. This looks very promising. However, there are new problems now and the amisslmaster.library is crashing badly now upon the initial "OpenLibrary()" call. This needs some investigation because there is no debug info appearing at all, I am afraid. Suggestions?

[jens-maus]

After some more investigation together with @cahirwpz we found out that the actual reason for the recent crashes with the updated toolchain was that the amigaos-cross-toolchain was using binutils 2.9.1 per default which didn't include fixes for proper baserel support. Now the amigaos-cross-toolchain has a --binutils=2.14 option to generate a cross compiler with the binutils 2.14 patchset (https://github.com/adtools/amigaos-binutils-2.14) including important fixes by Gunther Nikl.

After using these binutils sources with the update toolchain for putting const data in .text section, the map file for the amissl_vXXX.library binaries show:

.text          0x0003e604     0x5730 ./build_os3/openssl/libssl.a(t1_lib.o)
               0x0003ebb4                tls1_shared_curve
               0x00043114                SSL_get_shared_sigalgs
               0x00041f7e                ssl_prepare_clienthello_tlsext
               0x00042ed0                tls1_save_sigalgs
               0x0004334c                tls1_set_sigalgs
               0x000439d6                SSL_check_chain
               0x0003e972                tls1_ec_curve_id2nid
               0x0003ee5e                tls1_set_curves_list
               0x0003e9ac                tls1_ec_nid2curve_id
               0x0003f4ea                ssl_cipher_disabled
               0x0003f288                tls12_check_peer_sigalg
               0x00042cdc                tls12_copy_sigalgs
               0x0003e6cc                TLSv1_2_enc_data
               0x00042f62                tls1_process_sigalgs
               0x000423ea                tls_check_serverhello_tlsext_early
               0x00042376                ssl_parse_serverhello_tlsext
               0x00042936                tls12_get_sigandhash
               0x0003f15e                tls1_check_ec_tmp_key
               0x0003eae8                tls1_check_curve
               0x00042a62                tls12_get_hash
               0x0003e798                tls1_clear
               0x0003e710                tls1_default_timeout
               0x0003f1ea                tls12_get_psigalgs
               0x00041f94                ssl_prepare_serverhello_tlsext
               0x0003e688                TLSv1_1_enc_data
               0x00043c10                ssl_security_cert
               0x00042c04                ssl_set_sig_mask
               0x00043ca8                ssl_security_cert_chain
               0x0003e624                TLSv1_enc_data
               0x0003e76c                tls1_free
               0x0004219a                ssl_check_clienthello_tlsext_late
               0x00040236                ssl_add_serverhello_tlsext
               0x0003e71e                tls1_new
               0x000417a6                ssl_parse_clienthello_tlsext
               0x00042030                ssl_set_default_md
               0x000439f6                ssl_get_auto_dh
               0x000430a6                SSL_get_sigalgs
               0x000434e0                tls1_check_chain
               0x000432f6                tls1_set_sigalgs_list
               0x0003f884                ssl_add_clienthello_tlsext
               0x0003ed20                tls1_set_curves
               0x0003f45a                ssl_set_client_disabled
               0x000420ba                tls1_set_server_sigalgs
               0x0004298e                tls12_get_sigid
               0x00043948                tls1_set_cert_validity

... 

 .text          0x00059984     0x2964 ./build_os3/openssl/libcrypto.a(c_skey.o)
                0x0005a184                CAST_S_table2
                0x0005b984                CAST_set_key
                0x00059984                CAST_S_table0
                0x0005ad84                CAST_S_table5
                0x0005b184                CAST_S_table6
                0x0005a584                CAST_S_table3
                0x0005a984                CAST_S_table4
                0x00059d84                CAST_S_table1
                0x0005b584                CAST_S_table7

previously with the old GCC 2.95.3 compiler suite many const data was put into the .data section instead:

.data          0x00151498      0x348 ./build_os3/openssl/libssl.a(t1_lib.o)
               0x00151520                TLSv1_2_enc_data
               0x001514dc                TLSv1_1_enc_data
               0x00151498                TLSv1_enc_data
...
.data          0x00153c78     0x2000 ./build_os3/openssl/libcrypto.a(c_skey.o)
               0x00154478                CAST_S_table2
               0x00153c78                CAST_S_table0
               0x00155078                CAST_S_table5
               0x00155478                CAST_S_table6
               0x00154878                CAST_S_table3
               0x00154c78                CAST_S_table4
               0x00154078                CAST_S_table1
               0x00155878                CAST_S_table7

Thus, the issue seem to be resolved now and AmiSSLv4 for OS3/m68k doesn't crash anymore.