dragoonaethis / coriolis

Small event management tool

Coriolis

A small event management and ticketing utility.

If you run an instance, join the User Group on Telegram!

Warning: The project is currently being migrated to a containerized deployment and development environment. New deployments are currently not recommended. The current main branch hosts the classic environment on a Python 3.12 host. Please upgrade to Caelondia if you are on an older version first.

• Sell various ticket types with badge personalization (nickname, avatar)
• Provide applications (for volunteering, vendors, stalls, etc) with customizable forms
• Lightweight event/global pages system w/ Markdown support (for ToS, policies, etc)
• Integrates with django-payments (currently Przelewy24, a Polish payment gateway)
• Available in English and Polish

Development

Requirements:

• Developed and tested on Linux (any distro - run under a Linux VM on other OSes).
• You'll need Python 3.12+ with Poetry.
• For the database, you'll need PostgreSQL 14+.
• For the cache and background task queues, you'll need Redis 6.0+.
• For the ticket generation, you'll need a working Docker or Podman installation.
• Optional: You might want to use Mailhog as a fake SMTP server for development.

Setting up the environment:

• Clone this repo and copy .env.dist into a new file for your dev/prod environment.
• Configure your environment per instructions in that environment file.
• Set the ENV_PATH path to your newly-created .env file or load its contents as environment variables.
• Optional: To create the .venv in the project directory, run poetry config virtualenvs.in-project true.
• Run poetry install to create a virtualenv with all project dependencies.
• Run poetry shell to enter the virtualenv (whenever it's in the repo or the Poetry venv cache).
• Run ./manage.py migrate to set up the database for the first time.
• Run ./manage.py createsuperuser to set up your first user account.

You're now ready to run the development server:

• You can either load .env contents as environment variables or point the ENV_PATH variable at it.
• Run ENV_PATH=path/to/your/.env ./manage.py runserver to start the development server.
• If you got the ImproperlyConfigured, this means the env vars from this file were not loaded.
• Run ENV_PATH=path/to/your/.env ./manage.py rundramatiq to start the background task queue.
• For non-production environments, the top bar will be tinted red.

Common development tasks:

• ./manage.py makemessages -l pl - generate translation PO files for the pl locale.
• ./manage.py compilemessages - compile all available PO files into MO files (used by the app).
• ./manage.py makemigrations - generate database migrations after introducing changes in our models.
• ./manage.py migrate - apply missing migrations to your currently-running database.
• ./manage.py collectstatic - generate complete contents of the static directory (required for prod).
• ./manage.py rundramatiq --reload - runs the task queue with a code autoreloader.
• ./upgrade.sh - pulls the current Git branch and runs common upgrade steps, then restarts the service.

Production

• Set up the .env with non-debug, production values as noted on the Django Checklist.
• Set up your PostgreSQL database with backups and no access from the internet.
• Get credentials for a proper production SMTP server to send mails from.
• Set up the virtualenv with ...in-project true, poetry install, poetry shell.
• Run ./manage.py commands: collectstatic, migrate and createsuperuser.
• Deploy Django with Gunicorn: https://docs.djangoproject.com/en/4.2/howto/deployment/wsgi/gunicorn/
• Run a reverse proxy with Nginx: https://docs.gunicorn.org/en/latest/deploy.html
• Set up a 2nd domain for user uploads (MEDIA_URL) and expose the MEDIA_ROOT contents there.
• Make sure your production server has HTTPS configured: https://certbot.eff.org/
• Lock down SSH access via pubkeys only
• Block any non-SSH/HTTP/HTTPS conns via your firewall (ufw/firewalld/iptables)

Check out the contrib directory for some handy scripts and configs.

coriolis's Issues

Convert email templates to HTML

They're all sent as raw text, which doesn't work so well when field labels contain HTML directly. Special characters also get encoded incorrectly. This needs to be fixed.

Badge Previews: Bust preview caching on updated tickets

• On Firefox et al, uploading a new image works fine and the preview updates immediately.
• On Chrome et al, the preview does not update due to caching (Ctrl-F5 fixes it).
• Bust caching on those images whenever the ticket gets updated.

Fix Redis-based locks in dramatiq-crontab

dramatiq-crontab does not clean up after itself if it crashes or gets stopped uncleanly. All subsequent launches will fail due to a persistent lock (which we have disabled for now). Fix that.

Move login/logout links to POST forms

Currently they're GET-based, that's no good.

Add notification channels

On the event site, when people check in on the front lines, we need to notify ID handlers in the back what's about to come. People could do that manually, sure, but we can also set up a background notifier that sends messages to Discord on each used ticket. (This is partially implemented - Dramatiq, wiki docs, some local code.)

Custom pages for the whole platform, not just an event

Global custom pages are really helpful as the mean to communicate the privacy policy, the terms of service and other, legally important, information.

Dynaforms: Add "foreign tickets" field

Allow pinning 3rd party tickets to a form. This should not reveal whether such tickets exist.

Disabling 2FA does not work

Happened on an account with enforced 2FA - investigate?

Add custom JS/CSS for dynamic forms

We want some forms to automatically show/hide optional fields based on checkbox states. Custom JS would help with that.

Convert the ticket registration form to a dynamic form with custom fields

Can be used for t-shirt sizing for VIP tickets, etc.

Implement the Crew Panel

• Search by Ticket Code (exact, not fuzzy), Name, Email or Phone Number.
• Use a Ticket on-site (with payment method/notes).
• Create a Ticket on-site (from allowed types).
• Metrics and Admin Ops.

Force display a ticket type

Currently, all ticket types are displayed in the ticket selector under a few conditions (OK'd for online sales, in deadline, has rels), but we sometimes want to force display a ticket type even if it will no longer be available for sale. Allow force displaying a type.

Set up online payments for tickets

• Let users pay for their tickets online, not just on-site.
• django-payments seems decent, there's also django-payments-przelewy24

Upgrade to Django 4.0 and Python 3.10

Dependency	Version	Django 4.0?	Python 3.10?
django-allauth	^0.48.0	✅	✅
django-allauth-2fa	^0.8	✅	✅
django-colorfield	^0.6.3	✅	✅
django-crispy-forms	^1.14.0	✅	✅
django-debug-toolbar	^3.2.4	✅	✅
django-environ	^0.8.1	✅	✅
django-money	^2.1.1	❌	❌
django-payments	^1.0.0	✅	✅
django-phonenumber-field	^6.0.0	✅	✅
Markdown	^3.3.6	✅	✅
Pillow	^9.0.0	✅	✅
crispy-bootstrap5	^0.6	✅	✅
gunicorn	^20.1.0	✅	✅ (latest gevent req'd)
psycopg2	^2.9.3	✅	✅
requests	^2.27.1	✅	✅
sentry-sdk	^1.5.4	✅	✅
whitenoise	^6.0.0	✅	✅

Notes for visitors: If a dependency is marked as ❓/❌ here, it does not mean it doesn't work - we just haven't tested it yet, and changelogs/support tables don't mention Django 4.0/Python 3.10 support.

After Django 4.0 migration, we need to run makemigrations once to create a no-op migration.

Bump upstream django-dramatiq-email for Django 4.2 and switch to the PyPI package

We're running on a fork right now, let's undo that.

Admin Panel: Filter all objects by Event

Either on each Admin page or by setting a global context to one of the events.

Put application contents into a JSON field, not a glued-together text field

• Application details are currently just glued-together "notes" put into a <pre> with some funny CSS to make it look decent. It'd be nice to format it with HTML properly.
• To do so, we need to store application details as JSON, not just glued-together fields, then using data from the application type, format the output accordingly.
• This would also enable nicer CSV export with separate columns for each field.

Allow selecting payment methods per ticket type

• Make a TicketType - PaymentProvider M2M relation.
• Make online payments redirect to a provider picker, or autoredirect to the first available if only one is enabled.
• Make the "pay on site" an explicit payment option, so that some ticket types (VIPs) may have mandatory online payment.

Consider implications for minors/medical data in the system

• We just don't store anything like that at the moment, but we could get rid of the Convention Card if we did.
• Medical data most likely needs extra security design (encrypt with pubkey on the server side, provision browsers for privileged people with priv keys for decryption?)
• Also needs a review pass on all public endpoints (maybe lock down all endpoints behind login_required, make a separate URL block for public ones only?)

Redirect to the org ticket list if using an org ticket

Feedback from @baatochan: If a ticket belongs to the org, accreditation panel was accessed from the org detail list and the current user has perms on crew_accreditation + crew_orgs, redirect back to the related org detail view instead of the accreditation front.

Convert the "is of age" checkbox to explicit yes/no for new tickets

Feedback from HeV: In the accreditation panel, if registering a new ticket, there's a checkbox to denote if the accredited person is of age. Some accreditation volunteers ignore it entirely - convert this into an explicit, required Yes/No checkbox to make it more obvious.

Send emails in a task queue

Occasionally, the SMTP server dies - and so do pages which need to send mails as a result of whatever they're doing. We should send emails on a background task queue to retry them later.

Improve the image picker

• Currently it's just a plain file picker.
• Would be nice to have a dynamic image picker with cropping.
• Needs per-event image ratio configuration.

Dynaforms: Add a "ticket" field

Allow pinning a specific ticket to the dynamic form via a custom ticket selection field.

Prevent www-data from calling out to Docker

The way ticket rendering is implemented now is that the Dramatiq service worker, running as www-data, invokes the Docker binary directly. Nothing wrong with that per se, except that if anything takes over www-data, they can call Docker, and so they can just mount, rip and tear whatever they want on the system. Figure out a way to strip that privilege from www-data cleanly.

Make allauth forms POST-friendly

All the ON_GET features should be disabled: https://django-allauth.readthedocs.io/en/latest/configuration.html

Add extra metadata to tickets and applications

• Timestamps (created/updated) on everything
• Creation source (admin/self-registration/crew panel)
• Tickets: Used (bool)

Figure out timezone support

• Timezone support disabled as of dc409a3
• There doesn't seem to be a reason we'd like to support these, as the tool generally aims at small events...
• Figure out a neat way to implement this (in Django 4.0, maybe), or decide to drop support entirely.

Validate the dynamic form template on save

We just assume the form is correct - add a validator that tries to construct a DynamicForm and fails if an exception is thrown.

Implement state deadlines and handlers

Some tickets need some sort of an action taken within a time period (for example, VIP tickets need to be paid for within a few days of their purchase).

Add a state deadline mechanism where a ticket will have a deadline + handler to execute when it's reached. Background workers should periodically collect and run triggered handlers.

Optimize event filtering in Admin

• Currently we just add a event__name relation, which kinda sucks.
• Let's create a filter class that gets all active events and lookup returns ID -> event name pairs.
• https://docs.djangoproject.com/en/4.0/ref/contrib/admin/#django.contrib.admin.ModelAdmin.list_filter

Configure CSRF protection

CSRF protection should be enabled, along with .env-configurable trusted origins.

Add Anymail as a supported email backend

Currently we're doing just SMTP via Gmail, but the next large event will outgrow that. Add Anymail with Azure ACS or AWS SES support for transactional mails.

Add the 'url' field to dynamic forms

Useful if users need to provide a URL, and an URL only - nice to have, not required at the moment though.

Implement the ticket transfer system

The flow should more or less look like:

• Ticket Owner initiates a Transfer Session (button in ticket details).
• Ticket Owner gets a link to the Transfer session (/transfer/{id}) and sends it to the Ticket Receiver.
  • Transfer Sessions are visible to all logged-in users with a session link.
• Ticket Receiver fills in the required data (full name and email address) and requests the transfer.
  • Once requested, the Transfer Session is visible only to the Owner and Receiver.
  • Email notification is sent to the Owner that a Transfer Session is ready.
• Ticket Owner approves the transfer.
  • Email notification is sent to the Owner and Receiver that a ticket transfer has been finished.

Allow customizing the ticket/badge after initial registration

Make org ticket statuses more useful

Feedback from @baatochan:

• If an org has at least one used ticket, make that fact visible on the org list. (Right now you can only see how many tickets are used in the org detail view, OR on the org list if all registered org tickets are used.)
• If an org has cancelled or otherwise unusable tickets, make such tickets not count for status checks in the org list.

Serve media from a subdomain

Currently all media is served from a /media path - we should use a separate user media domain with a proper CSP policy.

Send the admin URL for tickets that need org attention

• We currently send orgs an email for new tickets awaiting approval/rejection.
• Admins have to manually go into the admin panel to set new notes/status.
• Templatize and send the URL in that mail too.

Implement the deferred payment flow

The scenario:

• You pay for the ticket, everything works, yay
• P24 sends a notification to our gateway endpoint that it worked, our handler verifies/accepts the payment and marks it as confirmed.
• P24 redirects the user back to Coriolis, handler checks if the payment is confirmed and updates the ticket status accordingly.

In most cases, the event order is just that, but occasionally something goes wrong and the notification shows up late.

The problem is that it's the user's view handler doing the ticket status updates - if P24 notification shows up after the redirect, it never gets updated. While this can be fixed by clicking Pay Online again (and django-payments will just do the redirect from the 3rd step), it's not a great experience.

• An easy fix is to update ticket statuses in the gateway endpoint handler, but if the payment fails, nothing ever hits it. Not great.
• The 3rd step should redirect to a "waiting for payment" screen where we poll for ticket status updates for a few seconds.
• If that doesn't happen, a background task that asks P24 what's up should fire.

Translate 2FA pages

They're all in English, Polish 'em.

Make application feedback emails more descriptive and customizable

If someone submitted 2-3 applications for attractions, it's currently difficult to figure out what was actually accepted. Make the form name dynamic or more descriptive, plus enrich emails with some extra info.

Set up Approve/Reject actions on Applications

Orgs should have a simple Approve/Reject button in the Admin Panel on each Application in the Waiting for Organizers state.

Allow setting an alternative Reply-To address for application types

Some application types should have their Reply-To set a given org's inbox, not the global event one.

Migrate to Django 4.2

https://docs.djangoproject.com/en/4.2/releases/4.2/

Notable issues:

• Setting update_fields in Model.save() may now be required
• Upgrade the upstream django-dramatiq-email library for 4.2 compat
  • EmailBackend now verifies a hostname and certificates. If you need the previous behavior that is less restrictive and not recommended, subclass EmailBackend and override the ssl_context property.
• {% blocktranslate asvar … %} result is now marked as safe for (HTML) output purposes.
• The entire STORAGES migration, maybe?

Deprecations:

• Passing encoded JSON string literals to JSONField is deprecated
• The length_is template filter is deprecated
• The STATICFILES_STORAGE setting is deprecated in favor of STORAGES["staticfiles"]

Send emails on various events

Notifications should be sent for:

• Registration confirmation to the attached email address (no mails for on-site registrations)
• Status changed on ticket (-> paid, cancelled, confirmed by orgs)
• New application submitted (notify orgs)
• Application status changed (notify submitter)

Make the footer configurable

Footer text and button e-mail address is currently hardcoded, this needs to be configurable per-event.

Set up Django Admin filtering

• Filter tickets by (Event, Status)
• Filter applications by (Event, Status)

Client-size field validation

• Highlight fields in red whenever an invalid value is entered on the client-side
• Affects required fields left empty, phone numbers and email addresses.
• No clear way to integrate with Django forms, help needed!