liip / django-template

Project template for Django projects

Django project template

To create a new project using this template, do the following:

pip install cookiecutter
cookiecutter gh:liip/django-template

Go to http://my-project.docker.test/ and start hacking!

Troubleshooting

If you get the following error when running cookiecutter:

Traceback (most recent call last):
  File "/home/me/.virtualenvs/tmp-1081dbf5557421f7/bin/pip-compile", line 7, in <module>
    from piptools.scripts.compile import cli
  File "/home/me/.virtualenvs/tmp-1081dbf5557421f7/local/lib/python2.7/site-packages/piptools/scripts/compile.py", line 16, in <module>
    from ..repositories import PyPIRepository
  File "/home/me/.virtualenvs/tmp-1081dbf5557421f7/local/lib/python2.7/site-packages/piptools/repositories/__init__.py", line 2, in <module>
    from .pypi import PyPIRepository
  File "/home/me/.virtualenvs/tmp-1081dbf5557421f7/local/lib/python2.7/site-packages/piptools/repositories/pypi.py", line 10, in <module>
    from pip.req.req_set import RequirementSet
ImportError: No module named req_set

That's because your pip version is too old. Upgrade it either with your package manager, or by running pip install --upgrade pip.

Deploying your site

You can use the provided fabfile to deploy your site. Before using it, check the fabfile.py file and adjust the settings at the top of the file.

Once the settings are adjusted and the dependencies installed, you can use the bootstrap command that will create the directory structure, push the code to the server and deploy your project. For example, the following command will bootstrap your dev environment:

fab dev bootstrap

You'll have to run the bootstrap command for each of the environments you want to bootstrap. After the site is bootstrapped, subsequent deployments can be done with the deploy command:

fab dev deploy

django-template's Issues

Add flake8-isort to the requirements

Add Wagtail

Just like DjangoCMS, provide a template for Wagtail sites.

Replace Fabric3 by Fabric in the requirements

Since it now supports Python 3

Add .pycharm_helpers in gitignore

Use pipenv

And remove code that loads variables from envdir and use python-decouple to handle environment-specific settings: https://pypi.org/project/python-decouple/. This should also fix #23.

Use more efficient reloader in backend container

Django's dev server uses almost no CPU time when idle with the watchman reloader. However this requires the watchman binary and pywatchman library.

The watchman binary is packaged in debian bullseye, but not the current stable release (buster).
I tried copying the binary from a 3rd party image in the Dockerfile. It is ugly but it works:

COPY --from=workbenchdata/watchman-bin:latest-buster-slim /usr/bin/watchman /usr/local/bin/
RUN install -m777 -d /usr/var/run/watchman/

An alternative solution could be to use runserver_plus from django extensions with the watchdog reloader. It is less efficient (~2% of idle CPU usage on my machine), but easier to setup (we just need to add watchdog to dev requirements).

Use console e-mail backend by default in dev env

EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'

Fix postgres credentials in docker-compose.yml

The postgres image does not accept empty passwords anymore:

django-template/{{cookiecutter.project_slug}}/docker-compose.yml

Line 11 in 8df302c

POSTGRES_PASSWORD:

2020-02-17T13:12:11.756884209Z Error: Database is uninitialized and superuser password is not specified.
2020-02-17T13:12:11.756931646Z You must specify POSTGRES_PASSWORD for the superuser. Use
2020-02-17T13:12:11.756939680Z "-e POSTGRES_PASSWORD=password" to set it in "docker run".
2020-02-17T13:12:11.756946255Z
2020-02-17T13:12:11.756951738Z You may also use POSTGRES_HOST_AUTH_METHOD=trust to allow all connections
2020-02-17T13:12:11.756957449Z without a password. This is not recommended. See PostgreSQL
2020-02-17T13:12:11.756962959Z documentation about "trust":
2020-02-17T13:12:11.756968449Z https://www.postgresql.org/docs/current/auth-trust.html

We should either set POSTGRES_HOST_AUTH_METHOD: trust or set a password.

Feature: Add services on request

Ideally the startup script would ask whether we need the following services to be setup:

• celery
• sentry
• heroku (idea is to push heroku master and it just works)

Make sure that CWD is correct before executing tasks

Some tasks may fail in weird ways when the current directory is not the project directory.
Example: running fab deploy from a submodule will try to push the submodule itself as the project!

Cannot pass arguments to fabric tasks

Some tasks, such as import-db, can take command-line arguments, for example to specify the path to the dump file to load. But if you check the output of fab --help import-db you’ll see that the task doesn’t seem to accept any argument. That’s caused by the @remote decorator. I guess because the wrapper function takes *args and **kwargs, fabric cannot parse its signature.

I’m not sure if it can be solved. If there’s no other way, we could remove the @remote decorator.

Disconnect other clients from DB on import

import_db often fails because postgres refuses to drop the schema when other clients are connected.
We could run something like this before starting the import:

psql -c "select pg_terminate_backend(pid) from pg_stat_activity where datname='{db_name}';"

Fix duplicate translation target in Makefile

translations -> compile-translations

Webpack HMR / live reload with pontsun

We may need to set sockPort: 443 in the webpack dev server config to make HMR and live reload work with pontsun.

Add mailhog container

So we can easily preview HTML e-mails

static folder location issue

We have the static folder created at project/project/static, but in the config file we define project/static
(

django-template/{{cookiecutter.project_slug}}/{{cookiecutter.project_slug}}/config/settings/base.py

Line 119 in c06fd15

STATICFILES_DIRS = (

).

requirements files are not pip-compiled by cookie-cutter if pip version > 8.1.2

Vagrant provisioning fails with pip version 8.1.2 because cookiecutter could not compile de .in requirements files.

Execute pytest before flake8 (tox.ini)

Avoid flake8 canceling tests

restart_process does not work

Use uwsgi --reload /path/something.pid

Use ManifestStaticFilesStorage

Backport gitlab CI fixes from pipenv branch

issues with scripts in /scripts (naming and content)

Django version

In requirements/base.txt, we have django==1.9, in requirements/dev.txt, we have django==1.8.4.

Add black

Now that we use black in all new Swing projects, it seems useful to integrate it in the template.

Improvement: Have a test database ready with fixtures

It's be nice the django-template proposes a way to quickly create fixtures for the tests + have a separated test database.

Prevent conflicts between multiple containers with the same name on the same network

All containers connected to a specific network share a common DNS namespace, which may lead to confusing conflicts.
Exemple: my frontend container/service is connected to the default network AND the pontsun network. It tries to proxy requests to the backend container/service, but the backend domain name resolves to another backend container from another project that is also part of the pontsun network.

To prevent this issue, we could give an explicit name to the default network (e.g. myproject) and use FQDNs (e.g. backend.myproject) in configuration files such as webpack.config.js.

Npm run start => error on fresh setup

Fresh setup gives the following error when running npm run start

(ENV) vagrant@capstone:/vagrant$ npm start

[email protected] start /vagrant
NODE_ENV=development webpack-dev-server --hot

/vagrant/node_modules/webpack-cli/bin/config-yargs.js:89
describe: optionsSchema.definitions.output.properties.path.description,
^

TypeError: Cannot read property 'properties' of undefined
at module.exports (/vagrant/node_modules/webpack-cli/bin/config-yargs.js:89:48)
at Object. (/vagrant/node_modules/webpack-dev-server/bin/webpack-dev-server.js:84:40)
at Module._compile (module.js:653:30)
at Object.Module._extensions..js (module.js:664:10)
at Module.load (module.js:566:32)
at tryModuleLoad (module.js:506:12)
at Function.Module._load (module.js:498:3)
at Function.Module.runMain (module.js:694:10)
at startup (bootstrap_node.js:204:16)
at bootstrap_node.js:625:3
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] start: NODE_ENV=development webpack-dev-server --hot
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! /home/vagrant/.npm/_logs/2018-10-29T21_24_36_244Z-debug.log

Upgrade webpack-cli to latest version fixes the problem, but then no assets is served correctly (404 on .css and .js). I've no time for more investigation atm.

Add test.txt in requirements objects in Makefile

Our tox config assumes the existence of a test.txt requirements file, but the make requirements target does not include it.

Improvement: Env vars

Document or improve scrirpt for creating/updating environment variable