Comments (20)
The URL is at some point known in advance though. Your application passes the URL to libcurl so it would know the URL. My opinion is that seems like a niche to implement in your application. Though I agree maybe some special generic type defines could be helpful.
I'm not interested in taking this issue up really but I just wrote CURLPROTO_STAYSAME
to only allow the same protocol on a redirect. In other words you use http only http redirects are allowed, or you use https only https redirects are allowed. I can't imagine a wide use (I'm not filing a PR) it's more poc to help get the ball rolling.
Something like CURLPROTO_ENCRYPTED_ONLY ? maybe would be easier and simpler as a combination of the flags of encrypted protocols. What seems difficult is handling connections that are "upgraded" to encryption. You'd have to allow the unencrypted protocol and then make sure they upgrade.
from curl.
Hello Jay, many thanks for jumping in. I think your CURLPROTO_STAYSAME
patch would solve the problem the best way. It avoids any ambiguity in judging protocols by "safety", while resolving the root concern. ("upgrading" a non-encrypted protocol has no point and can be considered dangerous even.)
I like your second proposal CURLPROTO_ENCRYPTED_ONLY
as well, but it keeps part of the ambiguity (not all encryptions are created equal), so I'd personally prefer your first solution.
About it being niche request, I don't agree; if securing a transaction is important, protocol downgrade is a real concern and dealing with this currently needs extra logic on the caller side (parsing the protocol in the source URL and forming a protocol filter based on that). Such extra logic may not always be trivial to implement, especially in scripts. wget
features an --https-only
option to solve this problem in a simple way. It is analog to your 'staysame' proposal.
For these reasons, I hope to see your patch promoted to a PR and seeing it in a future curl version. In the meantime any opinion and discussion is welcome.
from curl.
CURLPROTO_STAYSAME
sounds good to me too!
from curl.
The scope seems pretty narrow to me. What can cause a follow besides an http or https response? Nothing, it appears, which I didn't realize until I was most of the way through implementing CURLPROTO_STAYSAME
. In that case it only keeps http as http and https as https. If the user requests an http resource that redirects to https why wouldn't they want to allow that?
Something that stops the downgrade might be better and by making it just another flag that can be used in combination we're back to Viktor's suggestion. If the user doesn't want ambiguity they'd have to know the protocols they're expecting though. Consider:
curl_easy_setopt(curl, CURLOPT_REDIR_PROTOCOLS,
CURLPROTO_HTTP | CURLPROTO_HTTPS | CURLPROTO_NO_DOWNGRADE);
And behavior like:
Since version X.X.X there is flag CURLPROTO_NO_DOWNGRADE that can be used to stop a redirect from a secure resource to an unsecure resource. For example, if you request http://foo, which redirects to https://bar, which redirects to http:// baz, the redirect to http://baz would not be allowed.
master...jay:CURLPROTO_NO_DOWNGRADE
from curl.
If the user requests an http resource that redirects to https why wouldn't they want to allow that?
My original proposal was about to allow that, even though it's not something secure to do because the redirection target can be spoofed in the plaintext phase. In such case it's best to switch the origin URL to a secure alternative to keep the complete redirection chain use secure protocols.
'NO_DOWNGRADE' also reintroduces the ambiguity regarding what counts as a "downgrade" (f.e. is it an HTTPS to FTPS redirection considered a downgrade?), this is well mitigated by the optional selection of allowed protocols, it's good idea. Though internally, curl will still have to maintain a protocol list sorted by "upgrade" order.
My preference is still towards 'staysame', but if you guys find it better to allow "upgrades" as well, it's also an alternative - it gives more control, but some of them will be unsafe, it's up to the user.
Another alternative could be called 'STAYSAFE' (or 'STAYENCRYPTED'), and it'd mean that an unencrypted protocol can be changed to anything, while an encrypted protocol can only continue as encrypted - with an optional selection of allowed encrypted protocols. This seems simple, with less ambiguity and it hints no false sense of security about upgrading unencrypted protocols.
from curl.
It is also a bit tricky to easily figure out what an encrypted protocol is and thus what the level of the redirected-to protocol is. Like FTP, POP3 etc have "explicit" TLS which means they will connect as normal FTP:// etc and only upgrade to TLS if asked to do so with libcurl options.
from curl.
Indeed, the ambiguity is there as well.
from curl.
As this is still "just" a feature request and nobody seems to grab it now, I'll close it. This is now mentioned in the TODO: http://curl.haxx.se/docs/todo.html#Refuse_downgrade_redirects
from curl.
@jay created patches for both the "staysame" and later for the "no-downgrade" implementation. What would it require to get one of them merged? (I'd still prefer "staysame" because it's safer and simpler)
no-downgrade: 409fc44
staysame: bc2a9db
from curl.
Viktor those are drafts that are more concept. To reopen would require a different project collaborator who wants to take this on for assignment and explore the issue. To reiterate I think this is just too niche. Most common scenario I see is user wants to stop https->http but you can use CURLPROTO_HTTPS
exclusive to do that or with the curl tool --proto-redir -all,https
. As you've pointed out though it's hardly generic.
from curl.
Most common scenario I see is user wants to stop https->http but you can use CURLPROTO_HTTPS exclusive to do that or with the curl tool --proto-redir https.
uhm... I mean --proto-redir -all,https
(equivalent to --proto-redir =https
). See manual for explanation of +
, -
and =
from curl.
Thanks for you reply Jay.
I'd like to reiterate that http->https is also unsafe. (not only https->http)
Hence the recent notion of introducing HSTS in various browsers and creating the protocol for it:
https://developer.mozilla.org/en-US/docs/Web/Security/HTTP_strict_transport_security
https://tools.ietf.org/html/rfc6797
HSTS is a wider topic, but the starting point is that http->https redirection is deemed insecure and something to avoid.
It'd be nice to see curl
to follow other HTTP clients and make it easy to avoid this and other security pitfalls. Currently there is no way to enforce secure behavior with a single set of curl
/libcurl
options, in this regard, yes, probably not too many people will bother. Implementing this feature is lightweight and without any significant future cost (as proven by your patch).
Please note that CURLOPT_REDIR_PROTOCOLS
is already a niche feature compared to CURLOPT_FOLLOWLOCATION
[1]. Giving it an extra - useful - option would be an opportunity to make this feature utilized or utilized more.
[1] Google returns 379000 hits for the latter and 4610 for the former.
from curl.
I certainly wouldn't mind supporting HSTS, but that would need to be an option that specifies where to save the "cache" for that information - it would be tricky to add just transparently and by default - those "other HTTP clients" you speak of are all browsers I take it?. HSTS is however a slightly different topic than the much simpler option this issue is about.
So, this is not shutting down this feature, just that I prefer not to have open issues lingering around if nobody is working on them, especially not for feature requests.
from curl.
Sorry for bringing in HSTS, it was solely to make a point on http->https "upgrade" being harmful. "other HTTP clients" are all browsers at this moment, yes (and a recent proposal to implement it in wget
, here: http://www.burgersoftware.es/gsoc2015.pdf). It's indeed a different topic altogether (being a cache or a hard-wired list pulled from Chrome, Mozilla or wherever else it develops).
Fully on the same boat about lingering Issues. What is unclear to me is what else work needs to be done here, and is there anything I could help to make this happen? We'd need a conclusion on 'staysafe' vs. 'no-downgrade', and certainly one about it being too niche or useful for inclusion, but the patches (to me) look complete, except regression tests.
from curl.
I got the impression none of the patches were suggested for real so I've not bothered about them, and @jay seems to have basically the same view. Sorry if that was wrong.
I'll welcome a full patch/pull-request.
from curl.
It's rather me having being slow understanding the difference between a "real" patch and the ones @jay created. In some spare I'll try finding out what's missing for the staysame proposal.
from curl.
@vszakats it was no pull request and it was not a posted patch, it was a link to a set of commits. A "real" patch would be a patch sent to the mailing list, a pull-request would appear as one here on github.
from curl.
As for HSTS, I created a new issue: #1026
from curl.
Is there any progress for now? I try to detect redirects to http which i want to forbid. It would be nice if this feature would be available and also a special exit code to detect if the error was caused by the redirection downgrade.
from curl.
My solution/workaround for now is to use curl option --proto-redir =https
(and the equivalent libcurl option curl_easy_setopt( curl, CURLOPT_REDIR_PROTOCOLS, CURLPROTO_HTTPS )
), wherever -L
is enabled and the initial URL uses https
.
[ When the URL is a variable and/or it happens to use http
, the allowed redir protocols can be extended with http
. This logic is not practical to implement when using curl from scripts though (as noted in OP). Still not a generic solution overall, but covers most practical use-cases with existing facilities. ]
from curl.
Related Issues (20)
- clang-cl warnings in 'vtls/bearssl.c'
- NTLM authentication failing on Linux unless --http2 is specified HOT 3
- HTTP/2 libcurl regression HOT 5
- clang-cl warning in 'http/clients/ws-data.c' HOT 4
- Change in behavior for --remote-header-name HOT 2
- HAVE_STRUCT_TIMEVAL fails on Ubuntu 22.04 when it shouldn't HOT 3
- Bad order of linker arguments HOT 6
- curl 7.88.1-10+deb12u5 fails to download with tftp HOT 5
- mbedtls: trace with double-lines HOT 3
- About "add_custom_target(man ALL DEPENDS ${man_MANS})" HOT 10
- Write function callback is called twice after resume transfer and return CURL_WRITEFUNC_ERROR HOT 19
- File descriptor leakage HOT 10
- File descriptor leakage with multi handle and c-ares HOT 14
- mutli-thread crash with curl_share setting CURL_LOCK_DATA_CONNECT? HOT 4
- Test cases sometimes timeout HOT 1
- aws-sigv4 failing to calculate the right signature when using "content-type: multipart/form-data" HOT 7
- utf8 in powershell core HOT 8
- Having error alert when ./configure curl in a folder with an ".app" extension on macOS 14 HOT 3
- Please support setting adjustment algorithm sequence HOT 2
- `curl_multi_perform` not updating `running_handles` after redirection success HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from curl.