I did this I have used GitHub as an example here, but GitLab and B

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

/cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url=

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

8.7.1: rustls fails to pull from GitHub about curl HOT 13 CLOSED

Kangie commented on May 23, 2024 1

8.7.1: rustls fails to pull from GitHub

from curl.

Comments (13)

cpu commented on May 23, 2024 3

So I think the question is why the pkg-config changes are changing the default ca cert bundle

I opened a PR (#13251) with a patch that seems to resolve the problem for me locally.

$ LD_LIBRARY_PATH=/tmp/librustls-0.13.0/lib ./configure --with-rustls=/tmp/librustls-0.13.0
<snipped>
configure: Configured to build curl/libcurl:

  Host setup:       x86_64-pc-linux-gnu
  Install prefix:   /usr/local
  Compiler:         gcc
   CFLAGS:          -Werror-implicit-function-declaration -O2 -Wno-system-headers
   CPPFLAGS:        -isystem /tmp/librustls-0.13.0/include
   LDFLAGS:         -L/tmp/librustls-0.13.0/lib
   LIBS:            -lrustls -lz

  curl version:     8.7.2-DEV
  SSL:              enabled (rustls)
<snipped>
  ca cert bundle:   /etc/ssl/certs/ca-certificates.crt
  ca cert path:     no
  ca fallback:      no
<snipped>

  WARNING:  rustls enabled but marked EXPERIMENTAL. Use with caution!
  
$ make
<snipped>

$ LD_LIBRARY_PATH=/tmp/librustls-0.13.0/lib ./src/curl https://example.com
<!doctype html>
<html>
....

Someone more familiar with m4 might want to double check this is the right fix. My longstanding feud with autotools has perhaps evolved into a phobia after this weekend 🤪

from curl.

Kangie commented on May 23, 2024 3

My longstanding feud with autotools has perhaps evolved into a phobia after this weekend 🤪

I share your sentiment. Thanks for looking into this while I slept off my Sunday!

from curl.

bagder commented on May 23, 2024 2

Can we go back to this: how on earth can you link something wrongly ? It feels like until we know what the error is, blindly reverting it to the previous state might fix it for the short term but it does not seem like a stable solution since we don't know what actually broke the build.

When I build curl with rustls (both current git) I can get HTTPS from github perfectly fine, but I build with it statically. I believe that is what the CI builds with rustls do as well.

from curl.

cpu commented on May 23, 2024 2

@correabuscar Good eye! That does seem to be the critical difference. If I manually specify a --cacert matching the removed default that I don't need with the static build, or the build w/ the pkg-config changes reverted, then a "broken" build begins to work:

$ LD_LIBRARY_PATH=/tmp/librustls-0.13.0/lib ./src/curl https://example.com
curl: (60) rustls_connection_process_new_packets: invalid peer certificate: BadSignature
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

$ LD_LIBRARY_PATH=/tmp/librustls-0.13.0/lib ./src/curl --cacert /etc/ssl/certs/ca-certificates.crt https://example.com
<!doctype html>
<html>
<head>
    <title>Example Domain</title>

<snipped>

So I think the question is why the pkg-config changes are changing the default ca cert bundle and it's not a question of linking issues (as bagder's intuition seemed to know all along 😆 )

Edit: I've also filed rustls/rustls-ffi#409 to look into the error rustls-ffi is returning. I expected this situation to provoke a much easier to diagnose error indicating an unknown issuer.

from curl.

cpu commented on May 23, 2024 1

@bagder We're debugging this over in rustls/rustls-ffi#407

My initial findings seem to point to it being an issue specific to dynamically linking librustls w/ curl.

from curl.

bagder commented on May 23, 2024 1

IMO, we might as well just stick to this issue now as it identifies the current situation.

It baffles me how we can break rustls by linking it wrongly?

from curl.

bagder commented on May 23, 2024 1

Also, it seems our test suite is not good enough on the TLS front since it did not detect this regression.

from curl.

bagder commented on May 23, 2024

/cc @jsha ideas?

from curl.

cpu commented on May 23, 2024

@bagder Our investigation in rustls/rustls-ffi#407 points to this being a continuation of the problem reported in #13200

I've closed the rustls-ffi issue. I recommend we close this issue as well and consolidate remediation in a re-opened #13200. WDYT?

from curl.

cpu commented on May 23, 2024

It baffles me how we can break rustls by linking it wrongly?

This is mysterious to me as well and I would like to understand more about what's going on. In general we're advertising the newly added dynamic linking support as experimental for this exact reason.

from curl.

kpcyrd commented on May 23, 2024

Copying over from #13200, I'd suggest reverting both 9c42098 and 647e86a.

I suspect the original Gentoo issue this was meant to fix may not be present anymore now that Gentoo has net-libs/rustls-ffi 0.12.1.

from curl.

Kangie commented on May 23, 2024

Copying over from #13200, I'd suggest reverting both 9c42098 and 647e86a.

I suspect the original Gentoo issue this was meant to fix may not be present anymore now that Gentoo has net-libs/rustls-ffi 0.12.1.

If we do that builds on musl might break, which was the initial driver for using pkg-config. The original m4 was not suitable (though I will revert and do some testing to validate this after a few more hours of sleep).

from curl.

correabuscar commented on May 23, 2024

There are some interesting diffs between the bad vs good dirs, on my Gentoo, but here's what jumps out to me:

diff -upr curl-8.7.1.BAD/temp/build-abi_x86_64.amd64.log curl-8.7.1.GOOD/temp/build-abi_x86_64.amd64.log
--- curl-8.7.1.BAD/temp/build-abi_x86_64.amd64.log	2024-03-31 21:43:34.320394425 +0200
+++ curl-8.7.1.GOOD/temp/build-abi_x86_64.amd64.log	2024-03-31 21:45:17.977062047 +0200
@@ -628,9 +629,9 @@ configure: Configured to build curl/libc
   Install prefix:   /usr
   Compiler:         x86_64-pc-linux-gnu-gcc
    CFLAGS:          -march=skylake -mtune=skylake -mprefer-vector-width=128 -O2 -pipe -frecord-gcc-switches -ggdb -fvar-tracking-assignments -fno-omit-frame-pointer -ftrack-macro-expansion=2 -fstack-protector-all -Wno-trigraphs -fno-schedule-insns2 -fno-delete-null-pointer-checks -D_FORTIFY_SOURCE=2 -rdynamic -flifetime-dse=1 -Werror-implicit-function-declaration
-   CPPFLAGS:        
-   LDFLAGS:         
-   LIBS:            -lnghttp2 -lpsl -lrustls -lzstd -lzstd -lz
+   CPPFLAGS:        -isystem /include
+   LDFLAGS:         -Wl,-O1,--sort-common,--as-needed,-z,relro -L/lib
+   LIBS:            -lnghttp2 -lpsl -lrustls -lpthread -ldl -lm -lzstd -lzstd -lz
 
   curl version:     8.7.1
   SSL:              enabled (rustls)
@@ -652,8 +653,8 @@ configure: Configured to build curl/libc
   Verbose errors:   enabled (--disable-verbose)
   Code coverage:    disabled
   SSPI:             no      (--enable-sspi)
-  ca cert bundle:   no
-  ca cert path:     
+  ca cert bundle:   /etc/ssl/certs/ca-certificates.crt
+  ca cert path:     no
   ca fallback:      no
   LDAP:             no      (--enable-ldap / --with-ldap-lib / --with-lber-lib)
   LDAPS:            no      (--enable-ldaps)

EDIT: for who wants the whole diff (this if after configure phase on Gentoo was run): https://gist.github.com/correabuscar/65ea0d516d74f444342969196f37be84

from curl.

8.7.1: rustls fails to pull from GitHub about curl HOT 13 CLOSED

Comments (13)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

Jobs