lfex / poise

An LFE Library for framework-agnostic generation of static HTML content

poise

An LFE Library for framework-agnostic generation of static HTML content

Contents

• About
• Prerequisites
• Usage
• Name
• License

About ↟

This project is getting an overhaul to allow for better control over rendered Markdown in static sites. Progress is being tracked here:

• poise redesign

poise aims to be as simple and unopinionated solution as possible to the problem of generating static site content in the LFE/Erlang/BEAM world of web development.

While early versions of the poise LFE library were inspired by the Clojure statis, in more recent years the Rust-based project Zola has informed our design decisiions (in particular the areas where we found it inflexible).

Prerequisites ↟

Before building with rebar3, one needs to have the following installed:

• gcc, make, and related tools (also git)
• The Rust build tool cargo preferably installed using rustup
• The Markdown AST-parser mdsplode (installed using cargo)
• The jq and oniguruma libraries (used for querying JSON)
• Erlang and rebar3

To set up the environment variables required by mdsplod, you need to follow the instructions provided here. Once that's done, you will be able to run the following:

cargo install mdsplode
rebar3 compile
rebar3 lfe repl

Usage ↟

You can start poise in the LFE REPL with the following:

(poise:start)

The following are useful to check that the Rust binary is working with LFE:

(sploder:version)
(sploder:ping)
(sploder:echo "this is a test")

You can use a test file to check out some of the content-parsing functionality:

(sploder:read 'md "priv/testing/learn.md"))
(sploder:frontmatter)
(sploder:query "'.children.nodes[] | select((.depth == 3) and .source == \"Getting Started\")'")

Name ↟

poise, n. ¹

1. a stably balanced state; equilibrium
2. easy self-possessed assurance of manner; gracious tact in coping or handling; the pleasantly tranquil interaction between persons of poise; a particular way of carrying oneself; bearing, carriage

License ↟

Copyright © 2017-2024, Duncan McGreggor

Apache License, Version 2.0

---

1 https://www.merriam-webster.com/dictionary/poise

poise's Issues

Devise the means by which poise will utilise md file tooling

Since this tooling will be a binary (written in Rust; see #11), probably best to run this as a managed process in a supervision tree with erlexec.

Tasks:

• Convert to OTP app (gen_server)
• Add instructions for setting up mdsplode
• Test mdsplode running in erlexec
• Add wrapper functions for all the mdsplode commands that will be used
• Add server functions for handling parsed JSON for each of these:
  • ping
  • version
  • echo
  • read md
  • query
  • show frontmatter
• Test querying
• Test capturing output

Depends on:

• oxur/mdsplode#10
• oxur/mdsplode#22

Convert site data structure to route

This function will enable lmug web applications to serve site content dynamically.

Depends upon #1.

Determine the data format parsed data will be shared with poise

Create new project logo

The v0.3 redesign warrants a refreshed look 😉

Set up for building/testing on Travis CI

Select markdown tooling

Possible desired outcomes:

• Incur as little additional infra needs as possible
• Failing an Erlang solution, find a single binary solution
• ✅ Failing the availability of an existing single binary solution, create a single binary solution

Define generation flow/lifecycle

The output of this ticket will be a file in ./docs detailing everything below.

Things to keep in mind/include while creating this doc:

• Content management:
  • Steps to add new site
  • Steps to add new "sub-site"
  • Steps to add new page, new sub-pages
  • How to add metadata
    • How to indicate which template will be used
  • Protected metadata keys
  • How to add custom metadata
• Template management
  • How templates are organised
  • Steps to create templates
  • Variables the templates have access to
  • How to reference maps, lists, etc.
  • How to iterate over values
• Content Generation
  • Order of operations when content it generated
  • How to compile templates
  • How to (re)generate content for a single template
  • How to (re)generate content for a directory (including all children)
  • How to (re)generate all content

Add ToC to README

Add simple server for generated content

Maybe barista? Maybe the simple server used in the 10,000 things LFE demo?

poise redesign

Problem Statement

The LFE website has been using Zola for many years, and while this has been great, there are things that we would change if we could.

Item 1 - Markdown AST

we'd like to make some significant changes to how Markdown is displayed in certain parts of the site and we don't have the level of flexibility we would like. For instance, taking the headings of a certain level and all their children and rendering those in different parts of a template. To do that more simply/generally, one would like to have access to the AST of the markdown and then be able to call useful Zola or Tera template functions/built-ins on the AST (or select parts of it).

Item 2 - Pages vs. Sections

The page/section distinction that Zola makes is a little bit arbitrary and mostly useful for Zola internals (it's able to do some clever parsing/rendering parallelisation due to this abstraction). However, the downside to the user is a fair bit of confusion and a lot of unnecessary code/template duplication.

Item 3 - Template Layout Complexity

We've never really had a clean Zola template setup (possibly the root cause being "Item 2" above) and we're not really sure how to fix that without spending a great deal of time on some cleanup can can't really justify the time expenditure when what we have seems to work (if it's awkward to maintain/edit).

Item 4 - Zola and/or Tera Extensibility

A nicely extensible system would allow for us to make the above changes to it and still use the core of said system. Neither Zola nor its chosen templating system Tera offer the degree of customisation we need to accomplish our aims.

Item 5 - Clean Multi-site Capability/Workflow

This is probably possible in Zola, we just never were able to figure out a good answer in the various times we tried investigating the matter. Essentially, we'd like to be able to statically generate completely different content, templates, look-and-feel, etc., under one domain (under arbitrary paths). This would allow us to support, e.g., a "main" site, a blog, language documentation, etc.

Proposal

Direction Change

This redsign effort intentionally moves away from the inspiration provided by the Clojure-based stasis project, and instead towards something more Zola-like, but with greater simplicity and more data.

Features

• Site generation:
  • driven by configuration
  • multi-site capable
• Page data:
  • markdown as source, including "front matter" (metadata)
  • parsed as HTML and AST (mdast)
• Template system:
  • erlydtl (this gives us template composition, etc., as well as custom tags/extensibility)
  • clear means of template selection in md files (e.g., front matter)
  • consistent set of variables every page has access to

Weaknesses

This approach has the following difficulties than we need to be aware of (only some of which may be surmountable):

• LFE/Erlang are not as fast as compiled Rust, so generating a site will take longer (large sites potentially much longer)
• Erlang does not seem to have a markdown parser that can generate mdast, so we will likely have to install and use third-party tooling via the CLI (or port calls to the OS)
• As such, we will in all likelihood have to essentially create an in-memory "db" for each file consisting of filesystem path, front matter metadata, raw markdown, markdown AST, and parsed HTML (these + configuration data will need to be passed to the compiled Erlydtl templates)
• Erlydtl compiles templates to Erlang bytecode, which may then have their renderfunction called to output file data ... this means there will be a compile for the templates and any custom Erlydtl tags, a compile for poise code, and then a "run" step that will need to call this code and all associated/required renderfunctions.

Work

• #15
• Core
  • Convert library to OTP app (gen_server)
  • Define top-level configuration
  • Establish convention for template directory layout
  • Establish convention for markdown dirs -> URL paths
  • Define page metadata
  • Devise a means of discovering a page's template if not defined (ancestor metadata search)
  • Define page variables
• Parsing
  • #11
  • #12
  • #13
• Data
  • #14
  • Devise a method of pulling in all page ancestor data without having to hold an entire site's data in-memory
  • Ensure that all data (configuration, page metadata, md-related data, etc.) are ready to be passed to Erlydtl templates
• Output
  • Establish the convention for ensuring all templates have their render functions called
  • Ensure that project's that have poise as a dependency can use it to easily statically generate content
• Development
  • Investigate notify support
  • Set up a dev server
  • Set up a watcher

Add badges to README

Create generate function

This function will:

• iterate through site data structure
• call the content-generating function in each path / function pair
• write the generated content to the given path, under a configured output directory

Select a means by which data will be stored and accessed for each page

Depends on:

• #12
• #13

Define site data structure

A site data structure will represent all content for a site. It should at the very least be composed of path / function pairs where the functions are responsible for ultimately generating the content that will be available at the paired path.

Update project for latest Erlang, LFE, hex.pm, etc.

Tasks:

• Update deps
• Switch CI/CD to GH Actions
• Update badges
• Publish v0.2 to hex.pm