Failed to access to GCP bucket about datamol

[rkakamilan]

@maclandrol @hadim

Thank you for conducting various reproduction experiments.

Based on your comments, I suspected that there might be an issue with my gcloud configuration. I performed the following steps, which resolved the problem. Firstly, on my PC, I executed gcloud auth revoke and then used gcloud config configuration delete to remove all unnecessary configurations. As a result, I was able to access the bucket. It appears that the problem stemmed from expired information that remained in my gcloud settings.

Thanks again for all your efforts on my issue.

[maclandrol]

Hello @rkakamilan,

Could you share your version of fsspec, gcsfs and the datamol packages you are using ?

If possible also share the output of mamba list | grep google

My hunch is that this is due to an ongoing issue with google authentication system. Could you try installing the following in your environment google-auth >=2.13.0 and see if it fixes it ?

[rkakamilan]

@maclandrol Thanks for your comments.

The results of mamba list | grep xxxx

google-api-core           2.11.0             pyhd8ed1ab_0    conda-forge
google-auth               2.18.0             pyh1a96a4e_0    conda-forge
google-auth-oauthlib      1.0.0              pyhd8ed1ab_0    conda-forge
google-cloud-core         2.3.2              pyhd8ed1ab_0    conda-forge
google-cloud-storage      2.9.0              pyh1a96a4e_0    conda-forge
google-crc32c             1.1.2           py310h6f7428f_4    conda-forge
google-resumable-media    2.5.0              pyhd8ed1ab_0    conda-forge
googleapis-common-protos  1.57.1             pyhd8ed1ab_0    conda-forge
libgoogle-cloud           2.10.0               h20eaa6e_0    conda-forge

fsspec                    2023.5.0           pyh1a96a4e_0    conda-forge
gcsfs                     2023.5.0           pyhd8ed1ab_0    conda-forge

I tried the below, but it didn't worked.

• my conda env again and installation by pip.
• downgrading google-auth to 2.13.0

[maclandrol]

@rkakamilan I am trying to reproduce your error, but I can't. Can you post your full conda env.yml file ?
Also are you using both gcloud and gsutil ?

[hadim]

@rkakamilan I cannot reproduce on my local machine, but it might be due to my local machine being logged with a gcloud account. So, I tried to reproduce in a GitHub Codespace where gcloud is not available and I cannot reproduce. Looking at the libs on the codespace session, all the versions are the same except for libgoogle-cloud (but I also cannot reproduce using the same version as you).

$ micromamba list | grep google
  google-api-core                2.11.0        pyhd8ed1ab_0          conda-forge
  google-auth                    2.18.0        pyh1a96a4e_0          conda-forge
  google-auth-oauthlib           1.0.0         pyhd8ed1ab_0          conda-forge
  google-cloud-core              2.3.2         pyhd8ed1ab_0          conda-forge
  google-cloud-storage           2.9.0         pyh1a96a4e_0          conda-forge
  google-crc32c                  1.1.2         py311h98db957_4       conda-forge
  google-resumable-media         2.5.0         pyhd8ed1ab_0          conda-forge
  googleapis-common-protos       1.57.1        pyhd8ed1ab_0          conda-forge
  libgoogle-cloud                2.8.0         hac9eb74_2            conda-forge