S3 (HTTPS) profile doesn't work unless HTTP attempt is made first

#18495 · open · 5 comments

View on GitHub ↗

Tom-TBT

**Describe the bug** Logging with the S3 (HTTPS) fails despite working credentials. Issue is solved after attempt connection with S3 (HTTP) profile. **To Reproduce** It happens with our custom S3 endpoint (NetApp StorageGRID), but I could suspect that the issue is not related to that. Steps to reproduce the behavior: 1. Restart CyberDuck 2. With the S3 (HTTPS) profile, try to connect setting up your endpoint and credentials 3. Login fails with error "The AWS Access Key Id you provided does not exist in our records ..." 4. Use the same setting for S3(HTTP) profile, abort at warning "Unsecured S3 connection" 5. Try again with S3 (HTTPS) and exact same settings → you are connected **Expected behavior** Logging with the S3 (HTTPS) profile should work right away **Desktop (please complete the following information):** - OS: Windows 11 - Version 9.5.4 **Log Files** [cyberduck_log_error.txt](https://github.com/user-attachments/files/32898861/cyberduck_log_error.txt) Thank you for the great tool

Comments

AliveDevil

I'd love to help here, but we don't have access to a NetApp StorageGRID, and the only other non-Amazon S3 storage I have available for testing is Ceph RadosGW - which works just fine with the S3 (HTTPS)-profile. From your description it sounds like some issue on the server side, where the HTTPS endpoint isn't fetching the S3 access keys, but works on some cached data of the HTTP-endpoint, which can fetch S3 access keys. Please check the appliance log, and consult the NetApp StorageGRID knowledgebase or reach out the NetApp support.

Tom-TBT

Thank you for the prompt reply. I should also have mentioned that I don't have this issue if I connect from other clients. On Windows, I did it with minio-client, and with s3fs-fuse on Linux. And I had no issue connecting to my URL with HTTPS. Does it change anything for your understanding of the issue? If not, then I'll ask my IT admins if they can do something on the server side.

AliveDevil

Well … should've looked more closely. The first connection attempt uses the "default"-profile in `%USERPROFILE%\.aws\config` and `%USERPROFILE%\.aws\credentials`, despite a user name being set: ``` 2026-10-01 11:33:31,447 [Thread-0] DEBUG ch.cyberduck.core.s3.S3CredentialsConfigurator - Load profiles from Local{path='~\.aws\config'} and Local{path='~\.aws\credentials'} 2026-10-01 11:33:31,448 [Thread-0] DEBUG ch.cyberduck.core.s3.S3CredentialsConfigurator - Reading AWS file Local{path='~\.aws\credentials'} 2026-10-01 11:33:31,449 [Thread-0] DEBUG ch.cyberduck.core.s3.S3CredentialsConfigurator - Reading AWS file Local{path='~\.aws\config'} 2026-10-01 11:33:31,449 [Thread-0] DEBUG ch.cyberduck.core.s3.S3CredentialsConfigurator - Configure credentials from basic profile default 2026-10-01 11:33:31,450 [Thread-0] DEBUG ch.cyberduck.core.s3.S3CredentialsConfigurator - Set credentials from profile default 2026-10-01 11:33:31,451 [Thread-0] DEBUG ch.cyberduck.core.preferences.PreferencesReader - Setting credentials for Host{protocol=Profile{parent=Profile{parent=s3, vendor=iterate GmbH, description=null}, vendor=s3-https, description=S3 (HTTPS)}, region='null', port=443, hostname='s3-location.subdomain.de', credentials=Credentials{user='', password='', tokens='TemporaryAccessTokens{accessKeyId='********', secretAccessKey='********', sessionToken='', expiryInMilliseconds=-1}', oauth='OAuthTokens{accessToken='', refreshToken='', idToken='', expiryInMilliseconds=-1}', identity=null, properties={role_arn=null, mfa_serial=null}}, uuid='4800eccb-38a7-421a-b2d5-44735bafefa5', nickname='null', defaultpath='null', workdir=null, custom=null, labels=null} to Credentials{user='', password='', tokens='TemporaryAccessTokens{accessKeyId='********', secretAccessKey='********', sessionToken='', expiryInMilliseconds=-1}', oauth='OAuthTokens{accessToken='', refreshToken='', idToken='', expiryInMilliseconds=-1}', identity=null, properties={role_arn=null, mfa_serial=null}} … 2026-10-01 11:33:37,264 [background-9] DEBUG ch.cyberduck.core.s3.S3Session - Connect with session tokens TemporaryAccessTokens{accessKeyId='********', secretAccessKey='********', sessionToken='', expiryInMilliseconds=-1} ``` The second connect using the S3-HTTP-profile has some sideeffects, that clear the AWS-config S3 profile setting. The third connect, now using the S3 HTTPS-profile, doesn't read the default AWS config, and successfully connects: ``` 2026-10-01 11:34:09,990 [Thread-0] DEBUG ch.cyberduck.core.s3.S3CredentialsConfigurator - Load profiles from Local{path='~\.aws\config'} and Local{path='~\.aws\credentials'} 2026-10-01 11:34:09,990 [Thread-0] DEBUG ch.cyberduck.core.s3.S3CredentialsConfigurator - Reading AWS file Local{path='~\.aws\credentials'} 2026-10-01 11:34:09,991 [Thread-0] DEBUG ch.cyberduck.core.s3.S3CredentialsConfigurator - Reading AWS file Local{path='~\.aws\config'} 2026-10-01 11:34:09,995 [Thread-0] WARN ch.cyberduck.core.s3.S3CredentialsConfigurator - No matching configuration for profile null in {default=com.amazonaws.auth.profile.internal.BasicProfile@1bb3412} 2026-10-01 11:34:09,996 [Thread-0] DEBUG ch.cyberduck.core.preferences.PreferencesReader - Setting credentials for Host{protocol=Profile{parent=Profile{parent=s3, vendor=iterate GmbH, description=null}, vendor=s3-https, description=S3 (HTTPS)}, region='null', port=443, hostname='s3.amazonaws.com', credentials=Credentials{user='HBT56P2XPO5HK4ML18PP', password='', tokens='TemporaryAccessTokens{accessKeyId='', secretAccessKey='', sessionToken='', expiryInMilliseconds=-1}', oauth='OAuthTokens{accessToken='', refreshToken='', idToken='', expiryInMilliseconds=-1}', identity=null, properties={role_arn=null, mfa_serial=null}}, uuid='4800eccb-38a7-421a-b2d5-44735bafefa5', nickname='null', defaultpath='null', workdir=null, custom=null, labels=null} to Credentials{user='HBT56P2XPO5HK4ML18PP', password='', tokens='TemporaryAccessTokens{accessKeyId='', secretAccessKey='', sessionToken='', expiryInMilliseconds=-1}', oauth='OAuthTokens{accessToken='', refreshToken='', idToken='', expiryInMilliseconds=-1}', identity=null, properties={role_arn=null, mfa_serial=null}} 2026-10-01 11:34:09,996 [Thread-0] DEBUG ch.cyberduck.core.s3.S3CredentialsConfigurator - Load profiles from Local{path='~\.aws\config'} and Local{path='~\.aws\credentials'} 2026-10-01 11:34:09,999 [Thread-0] DEBUG ch.cyberduck.core.s3.S3CredentialsConfigurator - Reading AWS file Local{path='~\.aws\credentials'} 2026-10-01 11:34:10,000 [Thread-0] DEBUG ch.cyberduck.core.s3.S3CredentialsConfigurator - Reading AWS file Local{path='~\.aws\config'} 2026-10-01 11:34:10,000 [Thread-0] WARN ch.cyberduck.core.s3.S3CredentialsConfigurator - No matching configuration for profile null in {default=com.amazonaws.auth.profile.internal.BasicProfile@2d1862d} 2026-10-01 11:34:10,001 [Thread-0] DEBUG ch.cyberduck.core.preferences.PreferencesReader - Setting credentials for Host{protocol=Profile{parent=Profile{parent=s3, vendor=iterate GmbH, description=null}, vendor=s3-https, description=S3 (HTTPS)}, region='null', port=443, hostname='s3.amazonaws.com', credentials=Credentials{user='HBT56P2XPO5HK4ML18PP', password='', tokens='TemporaryAccessTokens{accessKeyId='', secretAccessKey='', sessionToken='', expiryInMilliseconds=-1}', oauth='OAuthTokens{accessToken='', refreshToken='', idToken='', expiryInMilliseconds=-1}', identity=null, properties={role_arn=null, mfa_serial=null}}, uuid='4800eccb-38a7-421a-b2d5-44735bafefa5', nickname='null', defaultpath='null', workdir=null, custom=null, labels=null} to Credentials{user='HBT56P2XPO5HK4ML18PP', password='', tokens='TemporaryAccessTokens{accessKeyId='', secretAccessKey='', sessionToken='', expiryInMilliseconds=-1}', oauth='OAuthTokens{accessToken='', refreshToken='', idToken='', expiryInMilliseconds=-1}', identity=null, properties={role_arn=null, mfa_serial=null}} ``` Can you confirm, that your .aws\config or .aws\credentials contain the `default`-profile, and the configured credentials mismatch the one from NetApp?

Tom-TBT

> Can you confirm, that your .aws\config or .aws\credentials contain the default-profile, and the configured credentials mismatch the one from NetApp? I confirm, in .aws\credentials, I have the [default] profile with `aws_access_key_id` and `aws_secret_access_key`, but it's not the credentials for the NetApp StorageGRID.

AliveDevil

We'll have to discuss this in the team - however the workaround for now is to change the `.aws\credentials` from `[default]` to `[aws]` or similar, so Cyberduck isn't trying to authenticate using your AWS credentials. Though you'd have to specify `--profile aws` on the AWS cli.