build fails about pdfio.jl

[sambitdash]

@konichuvak, I will assume the libcrypto.so is not able to be built. If possible can you upload the following files here for me to take a look:

deps\deps.jl and deps\build.log.

I do not have a Mojave or Catalina machine with me to test. So can only suggest a few approaches so that you can work with them and get over the issue if possible. I possibly cannot fix this issue and test it.

One other option could be to install, OpenSSL version "1.1.0" or higher on the system and see if everything works after that.

While darwin kernel 18.6.0 has been used for the build env and found to be working, the 19.0.0 of Catalina has never been tested.

https://travis-ci.org/sambitdash/PDFIO.jl/jobs/607447452

[konichuvak]

@sambitdash thank you for a quick reply!

deps/deps.jl is completely missing while build.log contains the same error as above.

OpenSLL seems up-to-date

 openssl version
OpenSSL 1.1.1c  28 May 2019

As per travis logs, it seems like the build is being performed with 1.0.5. I went ahead and installed julia 1.0.5, but received another error:

(test) pkg> build PDFIO
  Building PDFIO → `~/.julia/packages/PDFIO/LF83Q/deps/build.log`
┌ Error: Error building `PDFIO`, showing the last 100 of log:
│       _ENGINE_get_pkey_asn1_meth_str in libcrypto.a(tb_asnmth.o)
│       _look_str_cb in libcrypto.a(tb_asnmth.o)
│       _equal_wildcard in libcrypto.a(v3_utl.o)
│   "_strncmp", referenced from:
│       _ASN1_STRING_set_default_mask_asc in libcrypto.a(a_strnid.o)
│       _mask_cb in libcrypto.a(asn1_gen.o)
│       _asn1_cb in libcrypto.a(asn1_gen.o)
│       _SMIME_read_ASN1 in libcrypto.a(asn_mime.o)
│       _slg_write in libcrypto.a(bss_log.o)
│       _CONF_modules_load in libcrypto.a(conf_mod.o)
│       _int_def_cb in libcrypto.a(eng_fat.o)
│       ...
│   "_strncpy", referenced from:
│       _BIO_ADDR_rawmake in libcrypto.a(b_addr.o)
│       _BIO_lookup in libcrypto.a(b_addr.o)
│       _EVP_set_pw_prompt in libcrypto.a(evp_key.o)
│       _OPENSSL_DIR_read in libcrypto.a(o_dir.o)
│       _openssl_strerror_r in libcrypto.a(o_str.o)
│       _TS_RESP_verify_response in libcrypto.a(ts_rsp_verify.o)
│       _X509_NAME_oneline in libcrypto.a(x509_obj.o)
│       ...
│   "_strrchr", referenced from:
│       _oid_module_init in libcrypto.a(asn_moid.o)
│       _BIO_parse_hostserv in libcrypto.a(b_addr.o)
│       _CONF_modules_load in libcrypto.a(conf_mod.o)
│   "_strspn", referenced from:
│       _PEM_get_EVP_CIPHER_INFO in libcrypto.a(pem_lib.o)
│       _v2i_IPAddrBlocks in libcrypto.a(v3_addr.o)
│       _v2i_ASIdentifiers in libcrypto.a(v3_asid.o)
│   "_strtol", referenced from:
│       _ENGINE_ctrl_cmd_string in libcrypto.a(eng_ctrl.o)
│       _v2i_TLS_FEATURE in libcrypto.a(v3_tlsf.o)
│   "_strtoul", referenced from:
│       _ASN1_STRING_set_default_mask_asc in libcrypto.a(a_strnid.o)
│       _parse_tagging in libcrypto.a(asn1_gen.o)
│       _bitstr_cb in libcrypto.a(asn1_gen.o)
│       _stbl_module_init in libcrypto.a(asn_mstbl.o)
│       _OCSP_REQ_CTX_nbio in libcrypto.a(ocsp_ht.o)
│       _v2i_IPAddrBlocks in libcrypto.a(v3_addr.o)
│   "_sysconf", referenced from:
│       _CRYPTO_secure_malloc_init in libcrypto.a(mem_sec.o)
│   "_syslog", referenced from:
│       _slg_write in libcrypto.a(bss_log.o)
│   "_tcgetattr", referenced from:
│       _open_console in libcrypto.a(ui_openssl.o)
│   "_tcsetattr", referenced from:
│       _read_string_inner in libcrypto.a(ui_openssl.o)
│   "_time", referenced from:
│       _ASN1_TIME_diff in libcrypto.a(a_time.o)
│       _bnrand in libcrypto.a(bn_rand.o)
│       _CT_POLICY_EVAL_CTX_new in libcrypto.a(ct_policy.o)
│       _OCSP_check_validity in libcrypto.a(ocsp_cl.o)
│       _rand_bytes in libcrypto.a(md_rand.o)
│       _RAND_poll in libcrypto.a(rand_unix.o)
│       _X509_cmp_time in libcrypto.a(x509_vfy.o)
│       ...
│      (maybe you meant: _X509_STORE_CTX_set_time, _X509_time_adj_ex , _X509_time_adj , _x509_check_cert_time , _TS_TST_INFO_set_time , _TS_RESP_CTX_set_time_cb , _X509_VERIFY_PARAM_set_time , _PKCS7_add0_attrib_signing_time , _TS_TST_INFO_get_time , _CT_POLICY_EVAL_CTX_set_time , _SCT_CTX_set_time , _SCT_get_timestamp , _CT_POLICY_EVAL_CTX_get_time , _SCT_set_timestamp , _X509_cmp_current_time , _X509_cmp_time , _ASN1_UTCTIME_cmp_time_t , _X509_VERIFY_PARAM_get_time , _x509_set1_time , _err_clear_last_constant_time )
│   "_vfprintf", referenced from:
│       _OPENSSL_showfatal in libcrypto.a(cryptlib.o)
│   "_write", referenced from:
│       _conn_write in libcrypto.a(bss_conn.o)
│       _dgram_write in libcrypto.a(bss_dgram.o)
│       _fd_write in libcrypto.a(bss_fd.o)
│       _fd_puts in libcrypto.a(bss_fd.o)
│       _sock_write in libcrypto.a(bss_sock.o)
│       _sock_puts in libcrypto.a(bss_sock.o)
│      (maybe you meant: _UI_method_get_writer, _UI_method_set_writer , _RAND_write_file , _PEM_write_bio_PKCS7_stream , _PEM_write_X509_AUX , _PEM_write_PrivateKey , _PEM_write_bio_PrivateKey_traditional , _PEM_write_bio_PrivateKey , _PEM_write_PKCS8_PRIV_KEY_INFO , _PEM_write_bio_PKCS8_PRIV_KEY_INFO , _PEM_write_PKCS8PrivateKey , _PEM_write_bio , _PEM_X509_INFO_write_bio , _PEM_write_bio_PUBKEY , _PEM_write_DHparams , _PEM_write_EC_PUBKEY , _PEM_write_bio_EC_PUBKEY , _PEM_write_bio_ECPrivateKey , _PEM_write_bio_ECPKParameters , _PEM_write_DSA_PUBKEY , _PEM_write_bio_DSA_PUBKEY , _PEM_write_bio_RSA_PUBKEY , _PEM_write_RSAPublicKey , _PEM_write_NETSCAPE_CERT_SEQUENCE , _PEM_write_bio_NETSCAPE_CERT_SEQUENCE , _PEM_write_PKCS7 , _PEM_write_PKCS8 , _PEM_write_DHxparams , _PEM_write_DSAparams , _PEM_write_ECPKParameters , _PEM_write_bio_PKCS7 , _PEM_write_DSAPrivateKey , _PEM_write_bio_RSAPrivateKey , _PEM_write_X509_REQ_NEW , _PEM_write_bio_X509_CRL , _PEM_write_bio_X509_AUX , _PEM_write_ECPrivateKey , _BIO_ctrl_get_write_guarantee , _SMIME_write_CMS , _PEM_write_bio_X509_REQ , _PEM_write_X509_REQ , _PEM_write_X509 , _PEM_write , _PEM_write_bio_DSAPrivateKey , _SMIME_write_PKCS7 , _BIO_meth_set_write , _PEM_ASN1_write , _PEM_ASN1_write_bio , _PEM_write_bio_CMS , _PEM_write_bio_X509_REQ_NEW , _CRYPTO_THREAD_write_lock , _PEM_write_RSA_PUBKEY , _TXT_DB_write , _PEM_write_bio_DHxparams , _PEM_write_bio_DSAparams , _PEM_write_PUBKEY , _PEM_write_bio_ASN1_stream , _PEM_write_X509_CRL , _BIO_meth_get_write , _PEM_write_bio_Parameters , _PEM_write_bio_PKCS8 , _PEM_write_bio_CMS_stream , _PEM_write_bio_DHparams , _PEM_write_PKCS8PrivateKey_nid , _PEM_write_bio_PKCS8PrivateKey , _SMIME_write_ASN1 , _BIO_write , _PEM_write_CMS , _PEM_write_RSAPrivateKey , _PEM_write_bio_X509 , _PEM_write_bio_PKCS8PrivateKey_nid , _PEM_write_bio_RSAPublicKey )
│ ld: symbol(s) not found for architecture x86_64
│ clang-4.0: error: linker command failed with exit code 1 (use -v to see invocation)
│ make[2]: *** [Makefile.shared:229: link_shlib.darwin] Error 1
│ make[2]: Leaving directory '/Users/vlad/.julia/packages/PDFIO/LF83Q/deps/src/openssl-OpenSSL_1_1_0k'
│ make[1]: *** [Makefile:630: libcrypto.dylib] Error 2
│ make[1]: Leaving directory '/Users/vlad/.julia/packages/PDFIO/LF83Q/deps/src/openssl-OpenSSL_1_1_0k'
│ make: *** [Makefile:133: build_libs] Error 2
│ [ Info: Attempting to create directory /Users/vlad/.julia/packages/PDFIO/LF83Q/deps/downloads
│ [ Info: Downloading file https://github.com/openssl/openssl/archive/OpenSSL_1_1_0k.tar.gz
│ [ Info: Done downloading file https://github.com/openssl/openssl/archive/OpenSSL_1_1_0k.tar.gz
│ [ Info: Attempting to create directory /Users/vlad/.julia/packages/PDFIO/LF83Q/deps/src
│ [ Info: Attempting to create directory /Users/vlad/.julia/packages/PDFIO/LF83Q/deps
│ [ Info: Directory /Users/vlad/.julia/packages/PDFIO/LF83Q/deps already exists
│ [ Info: Changing directory to /Users/vlad/.julia/packages/PDFIO/LF83Q/deps/src/openssl-OpenSSL_1_1_0k
│ [ Info: Changing directory to /Users/vlad/.julia/packages/PDFIO/LF83Q/deps/src/openssl-OpenSSL_1_1_0k
│ [ Info: Changing directory to /Users/vlad/.julia/packages/PDFIO/LF83Q/deps/src/openssl-OpenSSL_1_1_0k
│ ERROR: LoadError: failed process: Process(`make install`, ProcessExited(2)) [2]
│ Stacktrace:
│  [1] error(::String, ::Base.Process, ::String, ::Int64, ::String) at ./error.jl:42
│  [2] pipeline_error at ./process.jl:705 [inlined]
│  [3] #run#505(::Bool, ::Function, ::Cmd) at ./process.jl:663
│  [4] run(::Cmd) at ./process.jl:661
│  [5] run(::BinDeps.SynchronousStepCollection) at /Users/vlad/.julia/packages/BinDeps/ZEval/src/BinDeps.jl:521 (repeats 2 times)
│  [6] satisfy!(::BinDeps.LibraryDependency, ::Array{DataType,1}) at /Users/vlad/.julia/packages/BinDeps/ZEval/src/dependencies.jl:944
│  [7] satisfy!(::BinDeps.LibraryDependency) at /Users/vlad/.julia/packages/BinDeps/ZEval/src/dependencies.jl:922
│  [8] top-level scope at /Users/vlad/.julia/packages/BinDeps/ZEval/src/dependencies.jl:977
│  [9] include at ./boot.jl:317 [inlined]
│  [10] include_relative(::Module, ::String) at ./loading.jl:1044
│  [11] include(::Module, ::String) at ./sysimg.jl:29
│  [12] include(::String) at ./client.jl:392
│  [13] top-level scope at none:0
│ in expression starting at /Users/vlad/.julia/packages/PDFIO/LF83Q/deps/build.jl:76
│ Version Libz is: 1.2.11
│ Version Libz is: 1.2.11
│
│ Full log at /Users/vlad/.julia/packages/PDFIO/LF83Q/deps/build.log
└ @ Pkg.Operations /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1096

I will try to install on ubuntu machine and post an update.

[sambitdash]

deps/deps.jl is completely missing while build.log contains the same error as above.

Expected as the build.jl has failed.

OpenSLL seems up-to-date

 openssl version
OpenSSL 1.1.1c  28 May 2019

It seems like there is OpenSSL but it's not getting loaded. Try to write a small script using https://docs.julialang.org/en/v1/stdlib/Libdl/#Libdl.dlopen to check if the library can be loaded.

Ideally on Mac OS, SSL need not be built from souce:

PDFIO.jl/deps/build.jl

Line 26 in 838b866

libcrypto = library_dependency("libcrypto", aliases=["libcrypto", "libcrypto-1_1-x64", "libcrypto-1_1"], validate=validate_openssl_version)

should successfully provide a library to be used.

[sambitdash]

https://forums.developer.apple.com/thread/119429 talks about system OpenSSL load failure in Catalina Beta 3. Not sure if the issue persists in the release. There is also workaround suggested.

[sambitdash]

@konichuvak Please test with Julia 1.3 and the latest build of PDFIO (#75 or later not 0.1.8 or the trunk latest). Seems the combination is working with Darwin 19.0.0.

https://travis-ci.org/sambitdash/PDFIO.jl/jobs/617830803

My presumption is Julia 1.3.0 will be needed for Mojave or Catalina.

[konichuvak]

@sambitdash Confirming successful install on julia1.3 when pulling from master!

 git clone --depth=50 --branch=master https://github.com/sambitdash/PDFIO.jl.git
Cloning into 'PDFIO.jl'...
remote: Enumerating objects: 378, done.
remote: Counting objects: 100% (378/378), done.
remote: Compressing objects: 100% (208/208), done.
remote: Total 378 (delta 207), reused 295 (delta 162), pack-reused 0
Receiving objects: 100% (378/378), 289.27 KiB | 3.32 MiB/s, done.
Resolving deltas: 100% (207/207), done.
(base)
in ~/jl vladஃ v1.2.0 vlad🅒 base
 cd PDFIO.jl
(base)
in PDFIO.jl on  master vlad🅒 base
 /Applications/Julia-1.3.app/Contents/Resources/julia/bin/julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.3.0 (2019-11-26)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(v1.3) pkg> add .
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Updating git-repo `/Users/vlad/jl/PDFIO.jl`
  Updating git-repo `/Users/vlad/jl/PDFIO.jl`
 Resolving package versions...
  Updating `~/.julia/environments/v1.3/Project.toml`
  [4d0d745f] ~ PDFIO v0.1.8 ⇒ v0.1.8 #master (.)
  Updating `~/.julia/environments/v1.3/Manifest.toml`
  [4d0d745f] ~ PDFIO v0.1.8 ⇒ v0.1.8 #master (.)

julia> using PDFIO
[ Info: Precompiling PDFIO [4d0d745f-9d9a-592e-8d18-1ad8a0f42b92]
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
 Resolving package versions...
  Updating `~/.julia/environments/v1.3/Project.toml`
 [no changes]
  Updating `~/.julia/environments/v1.3/Manifest.toml`
 [no changes]
┌ Warning: Package PDFIO does not have OpenSSL_jll in its dependencies:
│ - If you have PDFIO checked out for development and have
│   added OpenSSL_jll as a dependency but haven't updated your primary
│   environment's manifest file, try `Pkg.resolve()`.
│ - Otherwise you may need to report an issue with PDFIO
└ Loading OpenSSL_jll into PDFIO from project dependency, future warnings for PDFIO are suppressed.
 Resolving package versions...
  Updating `~/.julia/environments/v1.3/Project.toml`
 [no changes]
  Updating `~/.julia/environments/v1.3/Manifest.toml`
 [no changes]

julia>

[sambitdash]

Closing the issue.