nickjj / docker-django-example Goto Github PK
View Code? Open in Web Editor NEWA production ready example Django app that's using Docker and Docker Compose.
License: MIT License
A production ready example Django app that's using Docker and Docker Compose.
License: MIT License
pip freeze is no longer used. Replaced by pipreqs.
Running docker compose up --build
fails for me with message:
=> ERROR [worker app 6/10] RUN chmod 0755 bin/* && bin/pip3-install 0.7s
------
> [worker app 6/10] RUN chmod 0755 bin/* && bin/pip3-install:
0.636 /usr/bin/env: ‘bash\r’: No such file or directory
0.636 /usr/bin/env: use -[v]S to pass options in shebang lines
------
failed to solve: process "/bin/sh -c chmod 0755 bin/* && bin/pip3-install" did not complete successfully: exit code: 127
I am on Windows. I started WSL from within powershell by running wsl -d Ubuntu
. Then I ran the docker compose command inside the directory of this cloned project.
Is this a WSL issue? I looked inside my Ubuntu WSL and the directory bin does not exist....
EDIT: I now realise this is the same error as in issue #45. The user there ran some dos2unix command to fix potential file ending issues originating from an text editor. Note that I have not opened or changed any file in a text editor yet! I cloned the repo, started my WSL, perfomed the copying of the .env file inside the WSL and then tried to compose...
Hi, I'm able to run the code using the great ./run command but how would I go about using VScode to debug step by step? Thanks
Hello,
I've followed your Running this app manual and changed the name of the project with the rename-project script. Afterward, I built the project and run the migration. So, to test everything I wanted to run the 3 suggested ./run commands but I get for lint and format the following outputs:
./run lint
./run format
./run: line 56: @: unbound variable
./run: line 61: @: unbound variable
I guess something went wrong. Do you have any suggestions?
Best regards
Hello,
I am running through the tutorial, but my docker compos build keeps on failing.
Command I run
docker compose up --build
output
[+] Building 240.1s (20/50) docker:default
=> [web internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 2.20kB 0.0s
=> [web internal] load .dockerignore 0.0s
=> => transferring context: 216B 0.0s
=> [css internal] load .dockerignore 0.0s
=> => transferring context: 216B 0.0s
=> [css internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 2.20kB 0.0s
=> [worker internal] load metadata for docker.io/library/python:3.12.1-slim-bookworm 0.6s
=> [js internal] load metadata for docker.io/library/node:20.6.1-bookworm-slim 0.6s
=> [js internal] load .dockerignore 0.0s
=> => transferring context: 216B 0.0s
=> [js internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 2.20kB 0.0s
=> [worker internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 2.20kB 0.0s
=> [worker internal] load .dockerignore 0.0s
=> => transferring context: 216B 0.0s
=> [worker assets 1/7] FROM docker.io/library/node:20.6.1-bookworm-slim@sha256:2dab2d0e8813ee1601f 0.0s
=> [css internal] load build context 0.0s
=> => transferring context: 7.77kB 0.0s
=> [js internal] load build context 0.0s
=> => transferring context: 7.77kB 0.0s
=> CACHED [worker assets 2/7] WORKDIR /app/assets 0.0s
=> ERROR [css assets 3/7] RUN apt-get update && apt-get install -y --no-install-recommends bui 239.5s
=> [worker app 1/10] FROM docker.io/library/python:3.12.1-slim-bookworm@sha256:123229cfb27c384ee1 0.0s
=> [web internal] load build context 0.0s
=> => transferring context: 7.77kB 0.0s
=> CACHED [web app 2/10] WORKDIR /app 0.0s
=> [worker internal] load build context 0.0s
=> => transferring context: 7.77kB 0.0s
=> CANCELED [web app 3/10] RUN apt-get update && apt-get install -y --no-install-recommends b 239.5s
------
> [css assets 3/7] RUN apt-get update && apt-get install -y --no-install-recommends build-essential && rm -rf /var/lib/apt/lists/* /usr/share/doc /usr/share/man && apt-get clean && groupmod -g "1001" node && usermod -u "1001" -g "1001" node && mkdir -p /node_modules && chown node:node -R /node_modules /app:
20.39 Ign:1 http://deb.debian.org/debian bookworm InRelease
40.41 Ign:2 http://deb.debian.org/debian bookworm-updates InRelease
60.43 Ign:3 http://deb.debian.org/debian-security bookworm-security InRelease
80.46 Ign:1 http://deb.debian.org/debian bookworm InRelease
100.5 Ign:2 http://deb.debian.org/debian bookworm-updates InRelease
120.5 Ign:3 http://deb.debian.org/debian-security bookworm-security InRelease
140.5 Ign:1 http://deb.debian.org/debian bookworm InRelease
160.5 Ign:2 http://deb.debian.org/debian bookworm-updates InRelease
179.4 Ign:3 http://deb.debian.org/debian-security bookworm-security InRelease
199.4 Err:1 http://deb.debian.org/debian bookworm InRelease
199.4 Temporary failure resolving 'deb.debian.org'
219.4 Err:2 http://deb.debian.org/debian bookworm-updates InRelease
219.4 Temporary failure resolving 'deb.debian.org'
239.4 Err:3 http://deb.debian.org/debian-security bookworm-security InRelease
239.4 Temporary failure resolving 'deb.debian.org'
239.4 Reading package lists...
239.4 W: Failed to fetch http://deb.debian.org/debian/dists/bookworm/InRelease Temporary failure resolving 'deb.debian.org'
239.4 W: Failed to fetch http://deb.debian.org/debian/dists/bookworm-updates/InRelease Temporary failure resolving 'deb.debian.org'
239.4 W: Failed to fetch http://deb.debian.org/debian-security/dists/bookworm-security/InRelease Temporary failure resolving 'deb.debian.org'
239.4 W: Some index files failed to download. They have been ignored, or old ones used instead.
239.4 Reading package lists...
239.5 Building dependency tree...
239.5 Reading state information...
239.5 E: Unable to locate package build-essential
------
failed to solve: process "/bin/sh -c apt-get update && apt-get install -y --no-install-recommends build-essential && rm -rf /var/lib/apt/lists/* /usr/share/doc /usr/share/man && apt-get clean && groupmod -g \"${GID}\" node && usermod -u \"${UID}\" -g \"${GID}\" node && mkdir -p /node_modules && chown node:node -R /node_modules /app" did not complete successfully: exit code: 100
Docker version 24.0.7, build afdd53b
Docker Compose version v2.21.0
Thanks for the help
After building this, and running ./run manage collectstatic
I've noticed there are four directories for static assets in the web container (in addition to /app/assets
), but only two are used:
/public
- always empty/public_collected
- fills with the assets after collectstatic
/app/public
- the static assets collectstatic
collects/app/public_collected
- always emptyI wonder if this means there are some unnecessary creations in the Dockerfile, but I'm not sure which you intended to be correct? Or have I misunderstood something?
The readme states that I need to execute "./run pip3:install" to update dependencies during development.
Should this only be run when you want to "lock in" your dependencies to your docker container? i.e. if I'm working in my pycharm IDE and adding new python modules via pip3 , running the "./run pip3:install" command takes time to rebuild everything (3-4mins). I assume there's no need to rebuild after each module install as this would be impractical.
Just want to understand this process as a docker noob.
cookiecutter-django includes sentry and mailhog for development. Perhaps these could be included in your setup?
I follow all the steps and I keep getting this error. I am using windows 11
[web app 6/10] RUN chmod 0755 bin/* && bin/pip3-install:
0.274 /usr/bin/env: ‘bash\r’: No such file or directory
0.274 /usr/bin/env: use -[v]S to pass options in shebang lines
failed to solve: process "/bin/sh -c chmod 0755 bin/* && bin/pip3-install" did not complete successfully: exit code: 127
On the latest main branch, if you edit an HTML file gunicorn isn't picking up the change.
Django 4.1.X with Python 3.11.X but it's also an issue with Python 3.10.X too.
I tried a bunch of different code editors because I've run into situations where Vim's way of saving a file made it incompatible with certain file watchers. It's an issue with all of them that I tried (nano, Vim and VSCode).
After changing watched files you should see something like this:
hellodjango-web-1 | [2023-02-18 00:22:45 +0000] [8] [INFO] Worker reloading: /app/src/config/gunicorn.py modified
hellodjango-web-1 | [2023-02-18 00:22:45 +0000] [8] [INFO] Worker exiting (pid: 8)
hellodjango-web-1 | [2023-02-18 00:22:45 +0000] [10] [INFO] Booting worker with pid: 10
The above sometimes works if you edit the src/templates/layouts/index.html
or src/pages/templates/pages/home.html
and always works if you edit a Python file such as any views.py
file. JS and CSS also get updated on change in development which is using esbuild and tailwind instead of gunicorn.
The appropriate env vars defined in the .env
file appear to be set in the container's runtime environment:
nick@kitt ~/src/github/docker-django-example (main) $ run shell
python@49922f0f9045:/app/src$ python3
Python 3.11.1 (main, Feb 4 2023, 11:23:15) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> from distutils.util import strtobool
>>> bool(strtobool(os.getenv("DEBUG", "false")))
True
>>> bool(strtobool(os.getenv("WEB_RELOAD", "false")))
True
>>>
I also confirmed this in the app by going to the src/config/settings.py
file and dropping in:
print(bool(strtobool(os.getenv("DEBUG", "false"))))
print(DEBUG)
The Docker Compose logs produce True
for both. That is what we want since both are set to true in the .env
.
The TEMPLATES configuration is:
TEMPLATES = [
{
"BACKEND": "django.template.backends.django.DjangoTemplates",
"DIRS": [os.path.join(BASE_DIR, "templates")],
"APP_DIRS": True,
"OPTIONS": {
"context_processors": [
"django.template.context_processors.debug",
"django.template.context_processors.request",
"django.contrib.auth.context_processors.auth",
"django.contrib.messages.context_processors.messages",
],
},
},
]
Is that not correct?
I also removed the assets
container thinking maybe esbuild 0.17.X is having an effect due to both containers sharing the same Docker bind mount and I wanted to reduce variables but the same problem happens. My other Flask example app is using the same version of gunicorn and esbuild and its HTML code reloading is working.
Little help?
Hi,
Fresh build, didn't modified anything.
Ubuntu 21.10
Docker version 20.10.12, build e91ed57
docker-compose version 1.29.2, build 5becea4c
worker_1 | [2022-03-01 10:38:31,399: CRITICAL/MainProcess] Unrecoverable error: TypeError('Channel._connparams.<locals>.Connection.disconnect() takes 1 positional argument but 2 were given')
worker_1 | Traceback (most recent call last):
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/kombu/transport/virtual/base.py", line 923, in create_channel
worker_1 | return self._avail_channels.pop()
worker_1 | IndexError: pop from empty list
worker_1 |
worker_1 | During handling of the above exception, another exception occurred:
worker_1 |
worker_1 | Traceback (most recent call last):
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/redis/retry.py", line 45, in call_with_retry
worker_1 | return do()
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/redis/connection.py", line 608, in <lambda>
worker_1 | lambda: self._connect(), lambda error: self.disconnect(error)
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/redis/connection.py", line 673, in _connect
worker_1 | raise err
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/redis/connection.py", line 661, in _connect
worker_1 | sock.connect(socket_address)
worker_1 | TimeoutError: [Errno 110] Connection timed out
worker_1 |
worker_1 | During handling of the above exception, another exception occurred:
worker_1 |
worker_1 | Traceback (most recent call last):
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/celery/worker/worker.py", line 203, in start
worker_1 | self.blueprint.start(self)
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/celery/bootsteps.py", line 116, in start
worker_1 | step.start(parent)
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/celery/bootsteps.py", line 365, in start
worker_1 | return self.obj.start()
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/celery/worker/consumer/consumer.py", line 326, in start
worker_1 | blueprint.start(self)
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/celery/bootsteps.py", line 116, in start
worker_1 | step.start(parent)
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/celery/worker/consumer/connection.py", line 21, in start
worker_1 | c.connection = c.connect()
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/celery/worker/consumer/consumer.py", line 422, in connect
worker_1 | conn = self.connection_for_read(heartbeat=self.amqheartbeat)
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/celery/worker/consumer/consumer.py", line 428, in connection_for_read
worker_1 | return self.ensure_connected(
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/celery/worker/consumer/consumer.py", line 454, in ensure_connected
worker_1 | conn = conn.ensure_connection(
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/kombu/connection.py", line 382, in ensure_connection
worker_1 | self._ensure_connection(*args, **kwargs)
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/kombu/connection.py", line 434, in _ensure_connection
worker_1 | return retry_over_time(
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/kombu/utils/functional.py", line 312, in retry_over_time
worker_1 | return fun(*args, **kwargs)
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/kombu/connection.py", line 878, in _connection_factory
worker_1 | self._connection = self._establish_connection()
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/kombu/connection.py", line 813, in _establish_connection
worker_1 | conn = self.transport.establish_connection()
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/kombu/transport/virtual/base.py", line 947, in establish_connection
worker_1 | self._avail_channels.append(self.create_channel(self))
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/kombu/transport/virtual/base.py", line 925, in create_channel
worker_1 | channel = self.Channel(connection)
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/kombu/transport/redis.py", line 679, in __init__
worker_1 | self.client.ping()
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/redis/commands/core.py", line 954, in ping
worker_1 | return self.execute_command("PING", **kwargs)
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/redis/client.py", line 1173, in execute_command
worker_1 | conn = self.connection or pool.get_connection(command_name, **options)
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/redis/connection.py", line 1370, in get_connection
worker_1 | connection.connect()
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/redis/connection.py", line 607, in connect
worker_1 | sock = self.retry.call_with_retry(
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/redis/retry.py", line 48, in call_with_retry
worker_1 | fail(error)
worker_1 | File "/home/python/.local/lib/python3.10/site-packages/redis/connection.py", line 608, in <lambda>
worker_1 | lambda: self._connect(), lambda error: self.disconnect(error)
worker_1 | TypeError: Channel._connparams.<locals>.Connection.disconnect() takes 1 positional argument but 2 were given
hellodjango_worker_1 exited with code 0
any clues?
Hi Nick,
I have came across an unexpected behavior where tailwindcss ... --watch
is not detecting changes, following are steps to reproduce:
no changes Rebuilding activity is observed in docker logs -f hellodjango_css_1.
Forked repo: terencetwuo@ea53c77
Any help is much appreciated.
I have put a PDB in my code to debug and am using docker attach to connect to the web container. I hit the breakpoint but cant step through the code...
Cloned the repository, ran the recommended setup instructions and then I got this:
./run flake8
./run: line 49: @: unbound variable
Hi, thanks for the great starter example for someone wanting to get his feet wet in Django.
I have followed the initial steps found in #Running this app.
And I'm facing this error:
Building worker
[+] Building 6.3s (18/22)
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 38B 0.0s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 35B 0.0s
=> [internal] load metadata for docker.io/library/python:3.10.2-slim-bullseye 5.5s
=> [internal] load metadata for docker.io/library/node:16.14.0-bullseye-slim 2.3s
=> [assets 1/7] FROM docker.io/library/node:16.14.0-bullseye-slim@sha256:f00e66f4e3d5f3cf1322049440f9d84c79462cf2157f6d9bac26cec8d31f950e 0.0s
=> [internal] load build context 0.3s
=> => transferring context: 3.03kB 0.2s
=> [app 1/10] FROM docker.io/library/python:3.10.2-slim-bullseye@sha256:6faf002f0bce2ce81bec4a2253edddf0326dad23fe4e95e90d7790eaee653da5 0.0s
=> CACHED [app 2/10] WORKDIR /app 0.0s
=> CACHED [app 3/10] RUN apt-get update && apt-get install -y --no-install-recommends build-essential curl libpq-dev && rm -rf /var/lib/apt/lists/* /usr/share/doc /usr/share/man && apt-get clean 0.0s
=> CACHED [app 4/10] COPY --chown=python:python requirements*.txt ./ 0.0s
=> CACHED [app 5/10] COPY --chown=python:python bin/ ./bin 0.0s
=> CACHED [assets 2/7] WORKDIR /app/assets 0.0s
=> CACHED [assets 3/7] RUN apt-get update && apt-get install -y --no-install-recommends build-essential && rm -rf /var/lib/apt/lists/* /usr/share/doc /usr/share/man && apt-get clean && mkdir -p / 0.0s
=> CACHED [assets 4/7] COPY --chown=node:node assets/package.json assets/yarn ./ 0.0s
=> CACHED [assets 5/7] RUN yarn install && yarn cache clean 0.0s
=> CACHED [assets 6/7] COPY --chown=node:node . .. 0.0s
=> CACHED [assets 7/7] RUN if [ "development" != "development" ]; then ../run yarn:build:js && ../run yarn:build:css; else mkdir -p /app/public; fi 0.0s
=> ERROR [app 6/10] RUN chmod 0755 bin/* && bin/pip3-install 0.3s
[app 6/10] RUN chmod 0755 bin/* && bin/pip3-install:
#11 0.310 /usr/bin/env: ‘bash\r’: No such file or directory
executor failed running [/bin/sh -c chmod 0755 bin/* && bin/pip3-install]: exit code: 127
ERROR: Service 'worker' failed to build : Build failed
Highlighting the error part will be
ERROR [app 6/10] RUN chmod 0755 bin/* && bin/pip3-install 0.3s
[app 6/10] RUN chmod 0755 bin/* && bin/pip3-install:
#11 0.310 /usr/bin/env: ‘bash\r’: No such file or directory
executor failed running [/bin/sh -c chmod 0755 bin/* && bin/pip3-install]: exit code: 127`
Hello,
Thanks again for the great repo. I have a question about how you setup postgres for this project. I don't have a fullstack/djano background and I was not able to find the answer anywhere I looked.
When doing the very first migration, the current setup does not recognize my local postgres database so it throws an error saying that my target db name, lets call it testdb
, does not exist. Even if I create this database in my local postgres server, is still not recognized.
It only works when I docker exec into the postgres docker image and create the database in the running psql instance.
I also tried updated the docker-compose file to point from "postgres:/var/lib/postgresql/data"
to my locatl data directory (/var/lib/postgresql/16/main
) but still the same problem.
do you have an idea why is this happening? is it possible to post it at my local postgres ?
edit:
im using ubuntu 22, postgres 16.2
First, Thank you for the awesome repo, really helpful.
I would suggest updating docker-compose subcommands to match the new docker version.
Docker Compose is now in the Docker CLI, try `docker compose up`
Is there any concern that the primary container is running as root? I know it is a container but I have heard that it is good practice to run as a non-root user.
Thanks for this great setup guide!
I was able to successfully change project names on an older build around the Django 4 update.
However on the latest, I can ./run manage migrate on the default name, but if I change project names, docker-compose up --build, then I run into a problem doing ./run manage migrate.
From Docker logs:
postgres_1 | 2022-01-11 21:42:09.081 UTC [33] FATAL: password authentication failed for user "new_project"
postgres_1 | 2022-01-11 21:42:09.081 UTC [33] DETAIL: Role "new_project" does not exist.
From shell stdout:
django.db.utils.OperationalError: FATAL: password authentication failed for user "new_project"
My .env has the:
export POSTGRES_USER=new_project
export POSTGRES_PASSWORD=password
The error in #9 occurs under Ubuntu 20, and likely similar platforms enforcing non-root write privileges for the active user, due to hardcoded uid 1000 in the container images pulled during build. This explains build errors like these:
hellodjango-js-1 | mkdir: cannot create directory '../public/js': Permission denied
hellodjango-js-1 |
hellodjango-js-1 | Task completed in 0m0.001s
hellodjango-css-1 | mkdir: cannot create directory '../public/css': Permission denied
hellodjango-css-1 |
hellodjango-css-1 | Task completed in 0m0.001s
Resolution is to chown ./public* prior to first build with docker-compose. Likely have to do with sudo if the active user doesn't have root privileges.
$ sudo chown 1000 {public,public_collected}
Hi,
Would using django as api and adding new container for spa be possible/easy with this setup?
Hi,
Great project, should save a lot of time! I'm trying to run on Ubuntu 20.04.3 LTS and I get the following error:
Step 10/12 : COPY --chown=node:node ../ ../
ERROR: Service 'css' failed to build : COPY failed: forbidden path outside the build context: ../ ()
Commenting out line 23 of the Dockerfile allows the build to finish
COPY --chown=node:node ../ ../
Perhaps I'm doing something wrong. Django starts up and everything looks good but I'm unsure the impact. Using WSL2 on Windows with the same OS works OK.
Thanks!
Hello Nick,
First off, thank you loads for the work you put into this bundle, you have inspired me greatly to dive into Docker and containerization.
The issue at hand relates to hot reloading while developing. I find that whenever I modify a *.py file the webapp reloads just fine, showing the changes instantaneously. This is not the case when its an *.html template being modified. The css and js watchers are up and running. I checked the tailwind.config.js file and its pointing to the app folder, recursively and gathering all *.html files, so I don't know where to look now.
Thank you
Really enjoying the quality of this, I was wondering if it'd be handy to have a LOG_LEVEL
env var (detached from DEBUG
) that would allow Django to output logs when running with gunicorn in production.
Hi @nickjj, thank you for making this repository. I would have a look at this repo for my reference.
I'm looking for a Django project example that uses connection pooling (pgbouncer or pgpool) but seems like this repo does not use this.
My concern is because Django's DB connection is created during the request and closed after returns the response. this based on the documentation https://docs.djangoproject.com/en/3.2/ref/databases/#general-notes
I think using connection pooling would make this project better for production-ready project reference.
Thank you in advance
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.