GithubHelp home page GithubHelp logo

wagtail-nest / wagtail-bakery Goto Github PK

View Code? Open in Web Editor NEW
170.0 19.0 39.0 167 KB

A set of helpers for baking your Django Wagtail site out as flat files.

License: MIT License

Makefile 1.99% Python 96.01% HTML 2.00%
django static-site-generator wagtail

wagtail-bakery's Introduction

Wagtail-bakery

A set of helpers for baking your Django Wagtail site out as flat files.

License: MIT Build Status Version Monthly downloads

Wagtail-bakery is built on top of Django bakery. Please read their documentation for detailed configuration and how to build default Django flat files. Yes. Wagtail-bakery is not limited to build Wagtail pages specifically, mixed content is possible!

Links

Security

We take the security of Wagtail, and related packages we maintain, seriously. If you have found a security issue with any of our projects please email us at [email protected] so we can work together to find and patch the issue. We appreciate responsible disclosure with any security related issues, so please contact us first before creating a GitHub issue.

If you want to send an encrypted email (optional), the public key ID for [email protected] is 0xbed227b4daf93ff9, and this public key is available from most commonly-used keyservers.

Features

  • Single management command that will build your Wagtail site out as flat files
  • Support for multisite, theming and multilingual setup
  • Support for i18n_patterns
  • Support for generating a static API
  • Ready to use Wagtail Buildable views to build all your (un)published pages at once (no extra code required!)

Supported Versions

  • Python 3.8 - 3.12
  • Django 3.2 - 4.2
  • Wagtail >= 4.1

We aim to support the Wagtail versions as supported by Wagtail (current LTS, current stable).

Django/Wagtail combinations as supported by Wagtail (for the Wagtail versions as defined above).

Browser support

We align our browser support targets with that of Wagtail. Have a look at the official documentation.

Installation

pip install wagtail-bakery

Add bakery and wagtailbakery to your INSTALLED_APPS setting.

INSTALLED_APPS = (
    # ...
    'bakery',
    'wagtailbakery',
)

Configuration

Define whether you want to build multiple sites or the default site (see examples for impact on directory output), by default this settings is False.

BAKERY_MULTISITE = True

Add the build directory where you want to be the site be built as flat files.

BUILD_DIR = '/tmp/build/'

As you may know with Django bakery, the trickiest part is to make your current models/pages buildable with Buildable views. As Django Wagtail uses only the Page model at their lowest level, you can use at least one of the already present Buildable views provided by Wagtail bakery.

Build all published public pages (use for production).

BAKERY_VIEWS = (
	'wagtailbakery.views.AllPublishedPagesView',
)

Build all published and unpublished public pages (use for staging/acceptance).

BAKERY_VIEWS = (
	'wagtailbakery.views.AllPagesView',
)

To build static JSON files representing your site's API, use the following views:

BAKERY_VIEWS = (
	'wagtailbakery.api_views.PagesAPIDetailView',
	'wagtailbakery.api_views.PagesAPIListingView',
	'wagtailbakery.api_views.TypedPagesAPIListingView',
)

The API views use Wagtail's V2 API module. To configure the data that is rendered by these views, please refer to Wagtail's V2 API configuration guide.

Usage

Build the site out as flat files by running the build management command.

manage.py build

If you want to check how your static website will look, use the buildserver command after you have build your static files once.

manage.py buildserver

Examples

In the examples directory you can find a Wagtail setup with fixtures for a single site as well as a multisite setup.

Create a virtualenv and go to one of the examples, you can use the Make command to install all requirements, load fixtures and run the server.

As described in the usage section, use manage.py build to build out the example as static files.

Build output with BAKERY_MULTISITE=True:

build/example.com/index.html
build/example.com/about/index.html
build/example.com/blog/index.html
build/example.com/blog/example/index.html
build/static/

Build output with BAKERY_MULTISITE=False (default):

build/index.html
build/about/index.html
build/blog/index.html
build/blog/example/index.html
build/static/

Troubleshooting

For issues please submit an issue on GitHub.

Development

Which version combinations to include in Github Actions test matrix?

In order to keep for CI build time from growing out of control, not all Python/Django/Wagtail combinations will be tested.

Test as follow:

  • All supported Django/Wagtail combinations with the latest supported Python version.
  • The latest supported Django/Wagtail combination for the remaining Python versions.

Releases

  1. Create a new branch (e.g. release/1.1.3) for the release of the new version.
  2. Update the version number in setup.py following Semantic Versioning.
  3. Update CHANGELOG.md.
  4. On GitHub, create a pull request and squash merge it.
  5. On GitHub, if this is a minor release bump (for example 1.1.0 or 1.2.0 but not 1.1.1, 1.2.3), create a stable/1.1.x branch from main.
  6. (Optional) Publish to TestPyPI if you need to verify anything:
    1. Create and push a tag following the pattern X.Y.Z.devN (for example 1.1.3.dev1).
    2. Follow the action progress for the Publish to TestPyPI workflow.
    3. Check the result on TestPyPI: wagtail-bakery.
  7. Publish to PyPI:
    1. Create and push a tag following PEP 440 – Version Identification Specification (for example 1.1.3 or 1.1.3rc1), except for the .devN suffix used for testing (see Publish to TestPyPI step above)
    2. Follow the action progress for the Publish to PyPI workflow
    3. Check the result on PyPI: wagtail-bakery
  8. On GitHub, create a release and a tag for the new version.

Credits

Thanks to @mhnbcu for bringing this idea up initially, and Django Bakery for providing the initial bakery package.

Thanks to all the contributors for their help.

wagtail-bakery's People

Contributors

aetherunbound avatar bertrandbordage avatar ckljohn avatar clintonb avatar falseperfection avatar gasman avatar kaedroho avatar katdom13 avatar loicteixeira avatar robmoorman avatar stormheg avatar stuaxo avatar tomdyson avatar yedpodtrzitko avatar zerolab avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

wagtail-bakery's Issues

Add Wagtail 2.0 compatibility

Given the small size of this module, it can stay compatible with Wagtail 1.x as well as 2.x using try…except ImportError.

Apart from the modules rename, I don’t think anything will block Wagtail 2.0 compatibility.

admin menu

A great feature would be to have an admin menu for baking a site out of the box (or a least a setup for it)

Implement Wagtail Python Package Maintenance Guidelines

see https://github.com/wagtail/wagtail/wiki/Python-Package-Maintenance-Guidelines

Since wagtail-bakery is a first-party package we should aim to implement these guidelines.

I've copied the guidelines here and added tick marks for what needs to be done.

Readme

  • Must be written in Markdown and named README.md
  • Must be visible on PyPI and renders properly
  • Must have badges linking to PyPI and license
    • License
    • PyPI
  • Should have a badge linking to online code coverage results
    • Note: badge is there, link is broken Removed the badge for now
  • Must link to the changelog
  • Must mention what versions of Python, Django and Wagtail are supported
    • Note: refers to 'current LTS, current stable' which can be quite outdated. Could be improved by including a table of supported versions.
  • Must include an installation guide. This could be an abridged version of what is in the docs, but it must link to the full guide in the docs if this is the case
  • Must link to full documentation (or, for small packages, documentation included directly in the readme)
    • Note: refers to examples in subdirectories of the repository. Could be improved.
  • Must link to the Github discussions board that should be enabled on the repo (see Support)
  • Must mention where to report security issues ([email protected])

Documentation

Note: Small packages may use their readme as documentation. These guidelines only apply when documentation separate to the readme exists.

  • Must be available on the web
  • Must be written in markdown
  • Must be linked to from readme
  • Must mention what versions of Python, Django and Wagtail are supported
  • Must have an installation guide and a usage guide
  • Should have reference or explanation, depending on the complexity of the package

Changelog

  • Must have a changelog named CHANGELOG.md
  • Should follow Keep a Changelog

Contributing guide

  • Must have a contributing guide which is named CONTRIBUTING.md

License

  • Must have a permissive licence (such as BSD or MIT, but not GPL)
  • Should be licensed under BSD 3-clause, if possible
  • Must have a license file at the top of the repo

setup.py / setup.cfg / pyproject.toml

  • Dependency versions should be as wide as possible
  • Support for EOL versions of Python, Django and Wagtail should be removed from the earliest minor release of the package following the moment the dependency went EOL. This does not apply if provision is made to maintain support for the EOL dependency.
  • Testing and documentation requirements listed in extras
  • Must have classifiers for Python, Django and Wagtail versions
  • Must have classifier for license
  • Must have project URLs linking to Documentation and Changelog

Development Process

Branching

  • Default branch must be called main
  • Each major release must have a branch prefixed with “stable”. For example stable/1.0.x. This is to allow security fixes to be backported

Releases

  • Version numbers must follow PEP440
  • Each release must have a git tag and a github release
  • Each release must be mentioned in the changelog
  • Each release must be published to PyPI

PyPI

  • Package must have a PyPI page
  • At least two core team members must have admin access
    • Everyone who has permission to publish an official Wagtail package must have 2FA enabled on their PyPI account. Remember to check this when adding collaborators (this is visible in the PyPI web UI)

Automated testing

Django unit tests

  • Should use Django’s built-in test framework We use py.test, which is acceptable
  • Must have unit test coverage of at least 85%
  • Should aim for unit test coverage of at least 90%
  • Migrations and tests must be excluded from coverage reports
  • Must have instructions on how to run unit tests

Continuous Integration

  • Must have a Continuous Integration set up with Github Actions, Circle CI, or both
  • Should test against Wagtail nightly and report issues to #nightly-build-failures channel
  • Must be linted with flake8
  • Should be formatted with black and isort
    • Note: black is missing

Support

  • Should provide support through Github Discussions

Build fails if using external storage (such as S3)

I am using Amazon S3 (via boto3) to store the media files for a wagtail/django project. When I try to build my site, I get a ResourceNotFound error showing that it's looking for my S3 bucket inside my local directory, rather than remotely:

fs.errors.ResourceNotFound: resource '/Users/me/Documents/Django-Project/bakery/https:/s3-region-name-here.amazonaws.com/django-project-media/media' not found

The build command does work if I copy all of my media to my local directory, but it would definitely be ideal if I did not have to do that, especially since I intend to automate the build process. I acknowledge my use case is pretty unusual, so I understand if this is more of an enhancement request than a bug.

Error log attached below (some info changed for privacy reasons).


Full Error Log

(app) xxxxxxxxxx:Django-Project $ ./manage.py build
Traceback (most recent call last):
  File "/Users/me/.virtualenvs/Django-Project/lib/python3.6/site-packages/fs/osfs.py", line 324, in makedir
    os.mkdir(sys_path, mode)
FileNotFoundError: [Errno 2] No such file or directory: b'/Users/me/Documents/Django-Project/bakery/https:/s3-region-name-here.amazonaws.com/django-project-media/media'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./manage.py", line 12, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/me/.virtualenvs/Django-Project/lib/python3.6/site-packages/django/core/management/__init__.py", line 371, in execute_from_command_line
    utility.execute()
  File "/Users/me/.virtualenvs/Django-Project/lib/python3.6/site-packages/django/core/management/__init__.py", line 365, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/me/.virtualenvs/Django-Project/lib/python3.6/site-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/me/.virtualenvs/Django-Project/lib/python3.6/site-packages/django/core/management/base.py", line 335, in execute
    output = self.handle(*args, **options)
  File "/Users/me/.virtualenvs/Django-Project/lib/python3.6/site-packages/bakery/management/commands/build.py", line 109, in handle
    self.build_media()
  File "/Users/me/.virtualenvs/Django-Project/lib/python3.6/site-packages/bakery/management/commands/build.py", line 223, in build_media
    copy.copy_dir("osfs:///", smart_text(self.media_root), self.fs, smart_text(target_dir))
  File "/Users/me/.virtualenvs/Django-Project/lib/python3.6/site-packages/fs/copy.py", line 293, in copy_dir
    _dst_fs.makedir(_dst_path, recreate=True)
  File "/Users/me/.virtualenvs/Django-Project/lib/python3.6/site-packages/fs/osfs.py", line 327, in makedir
    raise errors.ResourceNotFound(path)
fs.errors.ResourceNotFound: resource '/Users/me/Documents/Django-Project/bakery/https:/s3-region-name-here.amazonaws.com/django-project-media/media' not found

Relevant Settings

# ----- settings.py -----

# Media Storage
MEDIAFILES_LOCATION = 'media'
MEDIA_URL = "https://s3-{}.amazonaws.com/{}/{}/".format(AWS_S3_REGION_NAME, AWS_STORAGE_BUCKET_NAME, MEDIAFILES_LOCATION)
DEFAULT_FILE_STORAGE = 'custom_storages.MediaStorage'

# Wagtail Bakery Settings
BUILD_DIR = os.path.join(BASE_DIR, 'bakery')
BAKERY_MULTISITE = True
BAKERY_VIEWS = (
    'wagtailbakery.views.AllPublishedPagesView',
)

# ----- custom_storages.py -----
from django.conf import settings
from storages.backends.s3boto3 import S3Boto3Storage

class MediaStorage(S3Boto3Storage):
    location = settings.MEDIAFILES_LOCATION

Build command is not localizing dates

When I view the pages through page preview they have localized dates, but the same doesn't happen when I build them. These are the relevant configurations:

LANGUAGE_CODE = 'pt-br'
LANGUAGES = (('pt-br', 'Brazilian Portuguese'),)
TIME_ZONE = 'America/Sao_Paulo'
USE_I18N = True
USE_L10N = True

Generated pages are blank when SECURE_SSL_REDIRECT is enabled

It is sometimes required to set SECURE_SSL_REDIRECT to True in the settings (e.g. you can't force the redirection before the request reaches Django). When you do so however, the generated pages will be completely blank.

This is because the request is not set as secure:
https://github.com/moorinteractive/wagtail-bakery/blob/4c2979956a9601ffd25811314e90b31f713e60c6/src/wagtailbakery/views.py#L91-L92

Which returns a HttpResponsePermanentRedirect which isn't handled (it reaches the 3rd if branch where reponse.content is b''):
https://github.com/moorinteractive/wagtail-bakery/blob/4c2979956a9601ffd25811314e90b31f713e60c6/src/wagtailbakery/views.py#L31-L40

While there might be some discussions to have on how a HttpResponsePermanentRedirect should be handled, I reckon this is for another issue and that the right fix here is to set the request as secure. I'll open a PR shortly.

Multi-site publish

I see that #1 was closed, however it still can be very useful for automatic publishing of multiple websites to different amazon s3 buckets

Also I am not sure how creating s3 redirects works with MULTISITE=True ./manage publish ?

Deprecation Warnings with Django 3.0, Wagtail 2.8 & Wagtail 2.9

Using Wagtail Bakery with Django 3.0, Wagtail 2.8 or Wagtail 2.9 produces some Deprecation Warnings. Some seem to be upstream so we might not be able to do much, but listing them here anyway.

  • Django 3.0
.../bakery/management/commands/build.py:133: RemovedInDjango40Warning: smart_text() is deprecated in favor of smart_str().
    self.build_dir = smart_text(self.build_dir)
  • Wagtail 2.8
.../wagtailbakery/api_views.py:8: RemovedInWagtail210Warning: PagesAPIEndpoint has been moved from wagtail.api.v2.endpoints to wagtail.api.v2.views and renamed to PagesAPIViewSet
    from wagtail.api.v2.endpoints import PagesAPIEndpoin
  • Wagtail 2.9
.../wagtail/core/middleware.py:8: RemovedInWagtail211Warning: wagtail.core.middleware.SiteMiddleware and the use of request.site is deprecated. Please update your code to use Site.find_for_request(request) in place of request.site, and remove wagtail.core.middleware.SiteMiddleware from MIDDLEWARES
    warnings.warn(
  • Python 3.3
.../wagtail/admin/rich_text/converters/html_ruleset.py:2: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
    from collections import Mapping

build result's root "index.html" results in Bad Request

I'm trying to automate publish to an s3 bucket, and am using the wagtail page_published signal to which connects to a function that uses, call_command() to call build.

This appears to work locally, but in my production environment on lambda/efs (via zappa/django-storages) the resulting index.html file seems to generate the following:

<!doctype html>
<html lang="en">
<head>
  <title>Bad Request (400)</title>
</head>
<body>
  <h1>Bad Request (400)</h1><p></p>
</body>
</html>

Any idea on how to debug this?

Clarify and simplify supported versions of Wagtail

At the moment, Wagtail Bakery supports Wagtail >= 1.6. While there are no real issues with supporting such an old version, it probably isn't the best use of resources (having to constantly ensure backward compatibility) and is probably worthless as it is unlikely that Wagtail 1.6 is still being used anywhere.

During the Wagtail core team meeting of September 12th 2018, it was decided that to only support the following versions: LTS version, current and current-1.

At the moment, it would mean 2.2, 2.1, 1.13 and 1.12 but with the next LTS version (2.3) around the corner and the breaking changes of Wagtail 2.0, it was decided that the package would drop compatibility with Wagtail prior to 2.0.

Unless someone makes a strong counter argument, this will be implemented in the next release.

TODO

  • Document supported versions of Wagtail
  • Update test matrix to test said versions

Add SitemapBuildableView

Sites need a sitemap and it'd be great to be able to generate this as part of the static build process.

I'm currently putting up a PR to answer this issue.

Add option to order pages during build

Hi and big thanks for your work on this beautiful tool!

I have a problem where the main page is built before the other pages. A link to a sub-page appears on the main page, but the new sub-page is not built yet. This results in 404s for a first dozen of seconds after a subpage gets published.

The solution might be to let users select how the pages will be ordered during a build. I would also suggest to build the latest published pages first, because menus/main pages tend to be created before all other pages.

What do you think? I only suggest this because I haven't found a workaround.

Wagtail 2 compatible release

Could you please release a Wagtail 2.x-compatible version? It looks like most of the work has been merged. Thanks!

Middleware isn't applied in get method

While using the custom get method, no Django middleware is applied. E.g. request.user.is_authenticated will not work in templates.

This method is overridden, since we want to let Wagtail's serve method handle the request and return a response object (so correct templates are returned, streamfields, etc.).

Nightly CI failure since Wagtail 2.16

Summary from @zerolab & @gasman

looks like this is due to https://github.com/palewire/django-bakery lacking Django 4.0 support - in fact it's not been updated since Django 2.1.

It probably wouldn't be a massive job to submit a 4.0 compatibility PR, although it would probably make sense to move them off travis-ci at the same time.

I'm also suspicious about why it only started failing after the 2.16 release, rather than as soon as we made dj4.0 the default in main - from the CI output it looks like it's installing from git correctly, but maybe there's something dodgy with the configuration.

Update - it's because it's installing wagtail/django from pypi and then installing wagtail main over the top (which would leave behind whichever of dj3.2 or dj4.0 was installed in the first step)

Does the django-bakery version still need to be be pinned at 0.12.7?

Thanks for this project - it's very handy!

wagtail-bakery currently requires django-bakery==0.12.3, which is now more than a year old.

Is it pinned to that because wagtail-bakery will break on higher versions, please? (Or is it just a very understandable matter of the maintainers being busy? 😄 )

Cheers
Steve

Towards 1.0

Discussed in #81

Originally posted by loicteixeira March 13, 2023
For the last release, @Stormheg asked if we wouldn't go 1.0, and I said not quite.
Once we merge #79 with Wagtail 4.2 support, we will probably do another release, but I still believe it's not quite ready for a 1.0 release.
However, it would be a good milestone to reach, so I'll try to list here was I think is needed (bottom line, it should support default Wagtail features)
Please comment on whether I'm missing something and/or we should drop some issues.

Supporting all Wagtail features

Bugs

Nice to have?

  • #4 (I have an idea to fix upstream)

Other

'HttpResponse' object has no attribute 'render'

Hi there,
Firstly, GREAT app and exactly what I had in mind with wagtail! Thank you.

Just found a small "bug" with the build management command when a content object returns an HttpResponse object that has already rendered

bakery/views/base.py

self.get(self.request).render().content

throws

AttributeError: 'HttpResponse' object has no attribute 'render'

The "fix" on my side was to

        resp = self.get(self.request)

        if hasattr(resp, 'render'):
            return resp.render().content
        elif hasattr(resp, 'content'):
            return resp.content
        else:
            raise Exception('Response has no content')

Which is a little ugly especially in the else case.. but I was just wanting to "make it work" on my side.

settings

BAKERY_MULTISITE = True

BAKERY_VIEWS = (
    'wagtailbakery.views.AllPublishedPagesView',
)
BUILD_DIR = '/tmp/asset-manager-build/'

Unicode characters in slugs

Bakery urlencodes URLs when creating folder names.
Which is indeed the URL asked by the browser to the HTTP server, but the HTTP server decodes the URL before searching the file in the system.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.