edemocracy / ekklesia-portal

Motion portal for the Ekklesia e-democracy platform

Ekklesia Portal

This repository is part of the Ekklesia e-democracy platform. It provides the motion portal Web UI, the public API and administrative interface.

You can find more development and operation documentation in the Ekklesia Documentation

Tech Stack

• Backend:
  • Main language: Python 3.11
  • Web framework: Morepath
  • Testing: pytest, WebTest
• Frontend
  • Templates PyPugJS (similar to Pug) with Jinja as template engine.
  • Sass Framework Bootstrap 4
  • htmx for "AJAX" requests directly from HTML.
• Database: PostgreSQL 15
• Dependency management and build tool: Nix
• Documentation: Sphinx with MyST Markdown parser.
• (Optional) Run on NixOS with the included NixOS module
• (Optional) Docker / Podman for running container images (built by Nix)

Development

To get a consistent development environment, we use Nix to install Python and the project dependencies. The development environment also includes PostgreSQL, code linters, a SASS compiler and pytest for running the tests.

Development Quick Start

This section describes briefly how to set up a development environment to run a local instance of the application.

Setting up the environment for testing and running tests is described in the section Testing in the Ekklesia documentation.

The following instructions assume that Nix is already installed, has Nix flakes enabled, and an empty + writable PostgreSQL database can be accessed somehow.

If you don't have Nix with Flakes support and or can’t use an existing PostgreSQL server, have a look at the Development Environment section in the Ekklesia documentation.

It's strongly recommended to also follow the instructions at Setting up the Cachix Binary Cache <https://docs.ekklesiademocracy.org/en/latest/development/dev_env.html#setting-up-the-cachix-binary-cache> or the first step will take a long time to complete.

1. Clone the repository and enter nix shell in the project root folder to open a shell which is your dev environment:

   git clone https://github.com/edemocracy/ekklesia-portal
   cd ekklesia-portal
   nix develop
   

2. Compile translations and CSS (look at dodo.py to see what this does):

   doit
   

3. Create a config file named config.yml using the config template from src/ekklesia_portal/config.example.yml or skip this to use the default settings from src/ekklesia_portal/default_settings.py. Make sure that the database connection string points to an empty + writable database.

4. Set up the dev database (look at flake.nix to see what this does):

   create_dev_db
   

5. Run the development server (look at flake.nix to see what this does):

   run_dev
   

Run help to see all commonly used dev shell commands.

Running In Production

A production environment can be built by Nix. The generated output doesn’t have additional requirements. The application can be run by a start script directly, using the included NixOS module or the Docker image built by Nix. Static assets are built separately and can be served by the included minimal Nginx. As for the application itself, we can build a standalone start script or a Docker image.

See the Ekklesia Operations Manual for more information.

History

Ekklesia Portal started as an improved implementation of Wikiarguments in Python 3.x using the Flask micro web framework. The project is now based on the Morepath web framework and tries to explore ideas from the Ruby project Trailblazer.

License

AGPLv3, see LICENSE

Active Authors

• Tobias ‘dpausp’
• Nico ‘kaenganxt’
• Holger ‘plexar’

ekklesia-portal's Issues

Share Propositions

We need a way to share links to propositions on social media (at least Twitter) or mail. UX should be similar to post sharing in Discourse.

Search improvements

• Filters values with multiple words: Support "" and ''
• Ignore case in search filters

Create proposition drafts from documents

We want an easy way to propose changes to official documents.

Workflow: select document -> select section in document for editing -> create proposition draft

Current Markdown contents of a document will be stored in the portal but should the document should be versioned elsewhere (preferably Git, we will add more integration with Git later).

Section are defined by attributes after headings using the syntax from pandoc / PHPExtra markdown: ### Section {data-section="1.1"}.
Editing a section includes its sub-sections.

Section identifiers should be shown in HTML as:

1.1.1 Section

Documents are created for a specific subject area (links it to a department). Propositions generated for it automatically use that subject area, too.

Proposition drafts should have a relation to documents via a change set entity that stores which part of a document is affected by the proposition.

Proposition redirect breaks for nonexistent proposition URLs

The path function must check if the proposition is None before proceeding.

Argument: Explain how "title" "abstract" "details" should be used

Fix Permissions of Subject Areas

Admins that have no Subject Areas subscribed can't edit Propositions with Subject Areas and Users and Admins with no Subject Area subscribed can't create a Proposition with a Subject Area.

Proposition Status DRAFT as External Link only

We can use the Proposition Status DRAFT as external link to discourse only, and so we still support rewriting the proposition content, abstract and motivation. Should have a Time Limit related to the policy in which the Proposition Status (automatically?) switches to Proposition Status Submitted and extracts the content, abstract and motivation from discourse via discourse-ekklesia plugin into the ekklesia portal.

Add script for organizing ballots

We want a script that reads a CSV file that specifies how to reorganize ballots. The script has an options to specify a voting phase and a department. Only propositions that are in the given voting phase are affected.

After running the script, all propositions in a CSV row should be in the same ballot. The first proposition in a row is the base proposition and following propositions have a "replaces" relationship with the first one.

Empty ballots should be deleted.

Example

Before

Ballot 1 -> WP001
Ballot 2 -> WP002
Ballot 3 -> WP003

CSV

WP001,WP002
WP003

After

Ballot 1 -> WP001, WP002 (replaces=WP001)
Ballot 2 deleted
Ballot 3 -> WP003

Autogenerate proposition voting ID

It should be possible to autogenerate proposition voting IDs when propositions are assigned to a voting phase from a string template. This is configured per proposition type and is optional.

Example: If the proposition is the first one of its type associated with a voting phase and the string template for the proposition type is "{abbreviation}{:03d}" with abbreviation set to "WP" the voting ID would be "WP001".

Spickerrrr propositions export

Export propositions in a format suitable for Spickerrrrr.....

Proposition edit form for admins

Admins should be able to edit propositions and see all fields.

Configurable top nav menu items

Admins should be able to configure which links are shown in the top nav menu. We could provide some built-in items and arbitrary URL items.

alembic for database migrations

We don't have a proper way to do database migrations yet.
Alembic provides database migrations based on SQLAlchemy models.

Add counter- and change propositions as drafts

Fix voting phases with no target date

• instead of showing no target date they show the target date with todays date

No new arguments when proposition is finished

what the title says :D

Openslides voting result import

We need a way to add voting results to the ekklesia portal:

Add a script that import the results from an Openslides motion export CSV, similiar to the organize_ballot script

Update proposition data from external sources

We already have a way to create new propositions from external sources ("importer"). Currently, this is implemented for Discourse posts. Proposition submitters and admins should be able to update propositions from an external source, overwriting data in the portal.

The first implementation should be able to fetch and parse Discourse posts but should make it not too hard to add support for different importer types.

Allow filters in search query

Add a syntax to allow using filters in search queries. For example: pirates phase:bpt201

Currently existing filters that can for now only be applied in the URL:

• Tags (tag)
• Voting phases (phase)
• Proposition types (type)
• Proposition status (status)
• Department (department)
• Subject area (subject_area)

Add fields from default query if simple search is used

We already have a default query for the proposition link in the top navbar. We should add it to simple search queries which don't include specific filters (just a string in the search box like someword without type:wp or similar)

Redirect all proposition URLs to the canonical URL

The proposition slug part of the URL behind the last / is ignored at the moment: every string is accepted and the proposititon is rendered according to the numerical ID. For example, a proposition with ID 1 and a title Full Title has the canonical URL example.com/p/1/full-title but example.com/p/1/just-a-random-string also works.

One object should always have only one URL, so we should redirect all URLs that differ from the full proposition URL to the canonical one.

Use of proposition types

• add to import scripts
• add to create proposition UI
• add to change proposition UI
• add to propositions in test db

voting result support

The Ekklesia Portal needs to support to display voting results.

Department field is wiped out and not selectable when editing a voting phase

Propositions: sort by date descending

Additional sort option for propositions. Can be sorted by LID because they include the timestamp at the start.

Openslides proposition export

We need a standalone script that exports propositions for an event (voting phase) in a format suitable for Openslides.

python export_openslides_csv.py -d Piratenpartei Deutschland" -v BPT19.2 > openslides_bpt192.csv

OpenID Connect support

Users should be able to login via OpenID Connect.

Advanced search form

Users should be able to search using a form which shows allowed values for each field that has fixed values (from enums like PropositionStatus or from the database, for example departments).

Submission workflow: draft, invite submitters, submit

1. User creates draft from empty proposition form or pre-filled with a document section.
2. Portal pushes draft to external discussion system.
3. User gets link with secret key to invite other submitters.
4. When required submitter count reached: show submit button to submitters.
5. User submits proposition from a form pre-filled with text from the external discussion system.

Proposition voting IDs

We need a proposition string field that is unique for a voting phase and proposition. It can be assigned if the proposition is associated with a voting phase. It can be an arbitrary string value without special meaning for ekklesia-portal. This helps us when we want to import propositions which already have some kind of identifier.

Piratenwiki proposition import

We need a standalone script that imports JSON dumps from the wiki.piratenpartei.de Antragsportal like:

python import_proposition_json.py --department=Bundesverband WP001.json WP002.json

Authors should be generated as dummy users.

Notification system

As a user I may want to get notifications for various events, for example:

• someone wrote an argument for/against my proposition
• someone supported my proposition (may be noisy)
• my proposition changed state
• admins have feedback about my proposition
• voting phase has started/will start in n days

We can use ekklesia-notify for that but we have to ask for contact addresses separately in the portal and store them. That differs from a voting component which only sends out notifications for currently logged-in users. For logged-in users, the encrypted recipient info from the OIDC provider can be used but we also want to send notifications to users that are offline.

Add more proposition filters

Currently, the propositions list supports a full text search and a tag filter. They can be combined. We want more filters like that.

• proposition status
• voting phase (name)
• proposition type (abbreviation)

Setting for supported languages

We need a config file setting for languages we want to support (displayed in the language switcher), like that:

app:
  languages: [de, en]

Make ekklesia auth data optional

We should accept OIDC users that don't have the expected fields verified and eligible yet and treat them as "unknown" or false depending on context.

Proposition state transition to qualified

SUBMITTED -> QUALIFIED when quorum for needed number of supporters is reached

Improve voting phase view

Voting phase attributes are simply displayed as a list at the moment. We want a more compact and nicer view, similar to how propositions are rendered.

Freeze supporting propositions and arguments, when proposition is finished

what the title says :D

Limit proposition actions to department members

Add a proposition action for requesting secret voting

Highlight supported propositions in list view

VVVote voting module integration

Provide full support for VVVote as first voting module. Department admins must be able to create a new election (voting) in VVVote from a voting phase in the portal. Multiple voting modules should be supported per department and admins should be able to choose where to create an external voting.

Automatically push drafts after some time

Add dropdown widget for proposition status

• the proposition status field only allows PropositionStatus enum values and therefore, when editing or creating propositions with another value, creates a bug.
• PropositionStatus.VOTING and higher status needs a proposition voting identifier
  • without it, it causes a bug #11
• PropositionStatus.FINISHED needs a voting result (with supported format) in the ballot
  • without it, it shows no voting result in the view (Should be avoided)

Subject area participation

Users must be able to participate in a subject area. The user profile should show subject areas for all departments of an user and allow changing the participation state.

We say that the user is an AreaMember when participating in a subject area.

Proposition shows current date for submitted and qualified date

Happens when the date is missing, I think.

Setting for a fallback language

We need a fallback language setting that is used if pages (Page) are missing for the currently selected language.

Personal proposition notes

Eligible voters should be able to save personal notes for propositions (questions, do I want to accept / decline?, ...)

Autogenerate ballot names from proposition voting identifiers

what the title says :D

Redirect to current page when switching languages