Default protocol for URLs with no protocol section is insecure about curl

[dfandrich]

On Thu, Feb 15, 2024 at 03:27:45AM -0800, SH wrote: How about a user or site configuration option to change the default, so that people can at least volunteer to suffer the pain of broken scripts in the name of security? Or even a build option to be used by security-above-all-else distributions. I think any site worth getting data from will give the same response on https anyway. Actually needing http seems kind of niche, and worth being explicit about.

Don't forget that curl is used plenty in automated ways within intranets and datacentres and single hosts where using https: doesn't provide much extra security. It's also used to contact proxy servers within a single host or LAN where the proxy server does the encryption. It may be niche in your case, but that's only one use case of many. The problem with adding an option is that in most cases, rather than changing the code to add an option you could simply change the code to use a secure, explicit URL instead. curl does not support a system-wide configuration file to add such an option, anyway. If you want to be sure that this isn't used, then you could always build and deploy a libcurl yourself that offers only what features you want. If you're happy changing behaviour for a specific users only, you could create a ~/.curlrc file and add some options to disable this behaviour. Some examples:

…

--proto-default https

--proto -all,https,ftps,imaps,ldaps,pop3s,smbs,smtps That would switch off http by default, as well as switching off support for all unencrypted protocols to boot (there are even more hardening options you could add in there if you're so inclined). I don't see that anything new needs to be added to libcurl to accomplish this that doesn't break backward-compatibility. Dan

[bagder]

This is by design, not a bug. We are not prepared to cause breakage to countless scripts in the world. We recommend you switch default proto or disable plain http in your .curlrc or similar.

I would welcome a deeper discussion on how, or when, this kind of change could be done, but that's a discussion. Not an issue.

[dfandrich]

This is historical behaviour. Changing it now will, unfortunately, undoubtedly break countless scripts out there in the real world.

[sh1boot]

How about a user or site configuration option to change the default, so that people can at least volunteer to suffer the pain of broken scripts in the name of security? Or even a build option to be used by security-above-all-else distributions.

I think any site worth getting data from will give the same response on https anyway. Actually needing http seems kind of niche, and worth being explicit about.

[sh1boot]

Well, when I say 'add an option' I don't mean an option for the diligent user (those users will, as you say, fix the URL by hand), but something to curb dangerous defaults. Your example ~/.curlrc entries are helpful, but they also break some normal and deliberate behaviour.

What I'm thinking is configuration around this block of code:

curl/lib/urlapi.c

Lines 1204 to 1220 in 5691a6c

	if((flags & CURLU_GUESS_SCHEME) && !schemep) {
	const char *hostname = Curl_dyn_ptr(&host);
	/* legacy curl-style guess based on host name */
	if(checkprefix("ftp.", hostname))
	schemep = "ftp";
	else if(checkprefix("dict.", hostname))
	schemep = "dict";
	else if(checkprefix("ldap.", hostname))
	schemep = "ldap";
	else if(checkprefix("imap.", hostname))
	schemep = "imap";
	else if(checkprefix("smtp.", hostname))
	schemep = "smtp";
	else if(checkprefix("pop3.", hostname))
	schemep = "pop3";
	else
	schemep = "http";

to make the default guesses the secure choices -- while still allowing guessing and still allowing deliberate use of http as appropriate.

That's the minimal-disruption solution I imagine which would allow safety-conscious distributions and site administrators to force-enable at build time, if they judge that the consequential pain (where people have to add http:// to some scripts if that's what they need) is worthwhile for broader security for their users.

As things stand, people out there on the internet still post bash <(curl -L trustworthy-site.com) commands for others to paste into their terminal, overlooking that while they may trust the site, curl's default behaviour opens this up to downgrade attacks and the code they run may well turn out to be malicious.

Even if it's off by default just having the option is an important first step. Having the option raises awareness.

[jay]

I don't agree that it is dangerous. If you don't select a secure protocol then that's what happens. I like Dan's suggestion to use --proto-default in curlrc to force it to https.

[sh1boot]

I mean.... somewhere low down on the first page of search results for 'curl pipe bash' I find this: https://www.arp242.net/curl-to-sh.html

Man-in-the-middle attacks: this is only a problem “if the developers omit the usage of TLS”, as the article already mentions. This is increasingly rare, and all of the cited examples use TLS.

They say that omission of TLS is "increasingly rare", but the syntax they give in their own example is: curl example.com/install.sh | sh which omits TLS because of the way curl behaves.

"Working the way you should have expected" is not the same as safe.

Just saying... 🤷