Cache not caching all deps? about deno_docker

[Maxim-Filimonov]

I don't know what does it do to docker image exactly but it completely ignores all the cache. The only thing I can think of is that .cache directory is used by pip too which in case of gcloud-sdk image might be something that google fiddles with at runtime.

FYI you can control the path of cache directory by DENO_DIR env var. Setting it to some different path might also work around the issue

That worked!
I'm not sure why google cloud run doesn't like default caching directory but it seems to be the case. As soon as I switched to different directory using:

# set DENO_DIR to avoid conflicts with google cloud
ENV DENO_DIR=./.deno_cache

It works like it suppose to.

[kt3k]

Sounds like a bug of Deno. https://deno.land/[email protected]/node/module_all.ts is the module used for enabling npm/node.js compatiblity.

The workaround would be adding the line RUN deno cache https://deno.land/[email protected]/node/module_all.ts after RUN deno cache **/*.ts

cc @bartlomieju

[bartlomieju]

This problem would be transparently solved by denoland/deno#16748, otherwise, we'd need to do some hacks to solve it. @dsherret any thoughts?

[Maxim-Filimonov]

Sounds like a bug of Deno. https://deno.land/[email protected]/node/module_all.ts is the module used for enabling npm/node.js compatiblity.

The workaround would be adding the line RUN deno cache https://deno.land/[email protected]/node/module_all.ts after RUN deno cache **/*.ts

cc @bartlomieju

That did help to certain extend. Got a new error now:

An npm specifier not found in cache: "ent"

What is strange I only get this error when building on Gcloud not when running a local docker build 🤔

[kt3k]

An npm specifier not found in cache: "ent"

What is strange I only get this error when building on Gcloud not when running a local docker build 🤔

Sounds like npm:ent is not cached for some reason only on Gcloud. What happens if you add RUN deno cache npm:ent command in your dockerfile?

[Maxim-Filimonov]

Apparently, there is some weird issue with Gcloud run. I don't know what does it do to docker image exactly but it completely ignores all the cache. The only thing I can think of is that .cache directory is used by pip too which in case of gcloud-sdk image might be something that google fiddles with at runtime.

I tried to download the same docker image that gcloud tries to run and it works perfectly without any cache issues on two different machines.
When tried to deploy that same instance to gcloud it fails miserably and cannot find any cache.

The workaround I have found is to bundle all dependencies using deno bundle inside docker image and run from .js file instead.
Probably, get the same result with less moving parts.

[kt3k]

I don't know what does it do to docker image exactly but it completely ignores all the cache. The only thing I can think of is that .cache directory is used by pip too which in case of gcloud-sdk image might be something that google fiddles with at runtime.

FYI you can control the path of cache directory by DENO_DIR env var. Setting it to some different path might also work around the issue

[kt3k]

I just realized that Dockerfiles other than bin- version already set ENV and probably don't have this problem. Do you have any specific reason to use bin instead of ubuntu, alpine, etc?

[inductor]

Sorry to jump in! I suspected it might be caused that you're on root user, so I tried to change the exec user as nonroot.

It worked!

I confirmed this is working on Cloud Run. I do not have a fully explanation here because I couldn't find a specific doc about user limitation on Cloud Run (I'm a Google Developer Expert but I'm mainly active in GKE and Kubernetes community.) But can you take a look if this approach works for you? @Maxim-Filimonov

Btw ADD is not Dockerfile bast practice so I changed it to COPY :)

ARG DENO_VERSION=1.30.0

FROM denoland/deno:bin-$DENO_VERSION AS deno
# debian based
FROM gcr.io/google.com/cloudsdktool/cloud-sdk:slim

ARG USERNAME=deno
ARG GROUPNAME=deno
ARG UID=1001
ARG GID=1001
RUN groupadd -g $GID $GROUPNAME && \
    useradd -m -u $UID -g $GID $USERNAME
USER $USERNAME
WORKDIR /home/$USERNAME/app

COPY --from=deno /deno /usr/local/bin/deno

# # These steps will be re-run upon each file change in your working directory:
COPY deps.ts .
RUN deno cache deps.ts

COPY . .

RUN deno cache **/*.ts

EXPOSE 8000

ENTRYPOINT ["/usr/local/bin/deno"]
CMD ["run", "--cached-only", "--allow-env", "--allow-read", "--allow-write", "--allow-run", "--allow-net", "./src/http_server.ts"]

[Maxim-Filimonov]

I just realized that Dockerfiles other than bin- version already set ENV and probably don't have this problem. Do you have any specific reason to use bin instead of ubuntu, alpine, etc?

I used bin based on this example -> https://github.com/denoland/deno_docker#using-your-own-base-image