statiqdev / statiq.web

Statiq Web is a flexible static site generator written in .NET.

Statiq Web

Statiq Web is a powerful static web site generation toolkit suitable for most use cases. It’s built on top of Statiq Framework so you can always extend or customize it.

📖 Read the documentation for updated quick start instructions as well as a more detailed guide.

💬 Use the Discussions repo for assistance, questions, and general discussion about all Statiq projects.

🐞 File an issue if you find a bug or have a feature request related to Statiq Web.

💙 Support the project if you find Statiq useful or need to purchase a private license.

Licensing

Statiq Web is dual-licensed under a public license (which is an open license such as MIT that limits commercial use to fewer than 10 total individuals working as employees and independent contractors and less than 100,000 USD total revenue in the prior tax year) and a private license (which is an open license such as MIT with additional limits on assignment and sublicensing).

Please see the licensing FAQs for more information.

statiq.web's Issues

Create an RSS feed module

Constructs an RSS feed from the input documents. How to handle feed items from multiple pipelines (might not need to worry about it if there's also a module that can "reuse" documents from other pipelines)? How to extract the excerpt (pull from metadata or provide a Func<IDocument, string>)?

Create a redirect module

Should be able to generate page redirects based on specified metadata values. Take a look at how Jekyll handles this in the jekyll-redirect-from plugin by using HTTP-REFRESH meta tags. Also note that there is support for this in GitHub Pages.

Implement external resource (file) watching

Need to figure out how a module can trigger watch events as necessary. For example, ReadFiles module should trigger for changes to read files while a database module could trigger for changes to tables, etc.

Implement caching for Razor compilations

Compiling Razor views for every page is getting to be a little slow. This is especially evident when using a layout for multiple views.

Publish Wyam as a dnx package

I'm still getting my head wrapped around using Wyam and what I'd like my workflow to be, but for now, I think I'd like to use Visual Studio to manage/edit my input project, using an empty asp.net 5 site.

With asp.net 5's project.json format having the ability to register commands, which can then be run from dnx or inside Visual Studio Code: Link. I think it would be helpful if Wyam could be packaged up and published as a DNX package.

That should let us register a command for wyam and integrate with the IDE or command-line a bit better.

All/most of the moving parts to do this might already exist, so this issue is either an enhancement request, or might just be a place to gather some of this documentation.

Create RSS Reader Module

Read RSS files and produces IDocuments with related meta tags for further processing.

This will allow Wyam to publish blogs based on RSS wire format. Support the outline extension as well (http://scripting.com/essayblog.xml)

Simplify command line arguments parsing

Nice project! May I suggest using Command Line Parser library to simplify argument parsing? https://github.com/gsscoder/commandline

If there was a reason for creating the method to parse the command line args the way its done in Program.cs, pls. ignore!

Make watching and rebuild dependent on modules

Right now --watch monitors file system changes. This can be expanded to watching a certain URL and trigger a pipeline when it changes (check etag).

Create a Mustachio module

https://github.com/wildbit/mustachio

Create a Split module

Takes input document content and splits it into multiple documents based on a delimiter.

Create a search module

Not sure how this one would work, but it would be cool to create a search index that could be queried at the client. Is there a JavaScript compatible Lucene client? The module should add content (and metadata?) to the search index and be usable from multiple pipelines.

Actually searching is probably the job of hand-written javascript code (but include examples in the docs).

Create OPML 2.0 Module

Read OPML outlines and produce documents with content and metadata.

http://dev.opml.org/spec2.html

Create a HTML minifier module

Create a HTML parsing module

A module that can parse HTML content and do something with it (like store results in metadata). Main use case is extracting the first paragraph from documents and storing it in metadata.

Implement recipes

Recipes should configure one or more pipelines. They should be contained in separate libraries and available via NuGet. Therefore, any interfaces need to go in Wyam.Abstractions.

They should also allow additional configuration as needed. Consider how they'll be configured from the command line.

Create a Pdf module

Convert HTML content to PDF

Create a link validation module

Crawls through specified HTML files and checks relative links to make sure they're all valid. Outputs warnings to trace. Does not perform any transformation.

Look at html-proofer: https://github.com/gjtorikian/html-proofer

Create a TypeScript module

Create a module that can take TypeScript content as input and generate JavaScript content as output.

Create a Contentful module

To pull content from https://www.contentful.com

Remove InternalsVisibleTo from Wyam.Core

It was added to support testing, but should be removed and all tests converted over to mocks, etc. if possible.

Create a Reprocess module

Module to reuse documents from previous pipelines. Not sure about the name... Example use case is pulling in blog entries from multiple pipelines to create an RSS feed.

Create a Join module

Module to perform concatenation on all input documents and generate a single output document (for example, to take a bunch of CSS files and output a single one). Allow and optional delimiter. Similar to string.Join(...) (as opposed to LINQ .Join(...)).

Implement a generic Metadata wrapper

Create a generic interface and class, IMetadata<T> and Metadata<T> that wraps Metadata and returns strongly typed metadata values. Use type converters or some other mechanism that will work in more cases than simple casting. I.e., if we have an int coded as a string object like Metadata["Foo"] = "123", then Metadata<int>["Foo"] should successfully return 123 as an int value. Create properties that create the wrapper to IDocument, the Razor page, etc. This should help minimize awkward metadata casting like ((string)Metadata["FileBase"]) == "Foo".

Make assemblies and namespaces declared in config available to Razor

Should include NuGet packages as well. For example, if Newtonsoft.Json is added as a NuGet package and the relevant namespaces are added in the config, then a Razor view should be able to use the JSON parser without any extra effort.

Use streams for document content

Instead of strings, use streams for binary content (see discussion in #25). This will allow documents to contain string or binary content. It should also yield better performance in some cases where transformations are optimized for streaming data (instead of having to read into a string). Will need to convert all existing modules over to streams. Should also add convenience getters and setters to convert from the stream from/to byte arrays and strings (in the case of getting data, handle reading the stream into the array and vise versa for setters).

Create an image manipulation module

It should be able to convert images as well as possibly create images from scratch (using SVG? System.Drawing?). Would be good for things like image-based text headers, etc.

Better NuGet handling

NuGet handling still isn't quite where I'd like it to be. Consider creating a global packages folder where the command line application is, and pre-propulate it with latest releases of common modules (Razor, Yaml, Markdown, etc.). Check the global for a match first, then check local, then download if not available. Also, don't update by default, only download if no match exists locally. Use a flag like --update-packages to actually update all packages (which might eliminate the need for the other control flag).

Give each document an ID

Create an IDocument.Id property that persists with the document (similar to metadata). This should generally be unique and can be used for things like trace messages, cache keys, etc. ReadFiles should set the ID to the RelativeFilePath. Consider other modules that clone extra documents - they should also set an ID. Should ID be populated from a Clone(...) overload? Log a warning if a duplicate ID is used. Find all trace message that refer to documents and use the ID (instead of path, etc.)

Create a JSON module

Should read JSON content and generate metadata. Probably based on Json.Net. Should also flatten the metadata like the YAML module does.

Create a TableOfContents module

Automatically extracts headings from a HTML document and generates a table of contents. How to specify where the ToC is inserted? How to identify whether a heading has an anchor (and what to do if it doesn't)? Should it also be added to the metadata? What if we just want it added to metadata without inserting the ToC (I.e., for display on another page)?

Possibly call this modules Headings instead of ToC (or maybe two different complimentary modules).

Implement support for partial views in Razor module

Should work just like it does in ASP.NET MVC.

Allow the Razor Module to have the ViewStart.cshtml file set rather than using a magic string

Currently the ViewStart used by the Razor Module is magic filename "_ViewStart.cshtml". Rather than using a magic filename, have it set via the constructor. This allows the more than just directory level organizing of the cshtml files. For example, on your site, rather than having a separate _ViewStart.cshtml in each subdir, you could have PostViewStart.cshtml, and you can specify a path to it, so you don't have to have such long Where predicates in your FileReader Module since all those files can live in a different subdir.

Add optional profile flag to get module timings

Not sure how deep this should go. A simple implementation would just send timings for each module and pipeline to trace. Maybe this should be done with verbose regardless (I.e., no flag to enable) - not sure the added overhead of setting timers adds enough to warrant turning on and off. If a flag is used, should it also be sent to modules or available in the execution context to modules can report in-module timing (I.e., compile vs. execute for Razor templates)? Maybe we need a new trace category for timing information.

Inspired from this PR in Jekyll: jekyll/jekyll#3762

Create a Liquid module

Takes Liquid template content and transforms it to output content. Use DotLiquid. Make the metadata, completed documents, and execution context available similar to how the Razor module does.

Create a Sass CSS module

Should read Sass CSS content and transform it to CSS content. Not sure if there's a suitable library available (if not, this will have to wait - not building a new Sass parser).

Add a global caching concept

There are several modules that would benefit from some sort of caching between executions:

• Razor compilations (but not executions) - already uses it's own cache
• Markdown
• Yaml
• Less

The cache should be keyed to some sort of hash on the input content since not all inputs are going to come from files, and even if they do they might be manipulated by other modules before getting to the one that needs caching. The key needs to be unique enough to prevent false positives. Should we first check for a matching hash and if one is found, compare the content for fast hit but then guaranteed no false positive? This would require storing the old content along with the hash and might take long enough that the cache is a moot point... Another requirement is that we can only cache stuff that can't refer to other metadata because we have no way of knowing what that metadata will be on each run (and it might be different).

The cache should be passed in the execution context and be available for any module to use (somehow keyed to the module to limit the search space?). Provide methods to generate cache entries from the document (automatically calculate hash, store content, etc) and check the cache from a document. Cache value should be a raw object so that things other than content (like Razor compilations) can be stored.

Create a CSS minifier module

Add better tracing to preview web server

For example, outputting a trace message when a file can't be found would be helpful in pointing out missing resources.

Rename Wyam.Abstractions to Wyam.Common

It's becoming clear that there may be additional code that module libraries might need beyond the interfaces currently in Wyam.Abstractions. Renaming will keep the spirit of the library while expanding the implied scope. Also better to do it early now then when there are actually third-party module libraries out there. The NuGet package under the old name will have to be delisted, or at least a comment added.

Make the cache thread-safe

It was always the intent that modules could execute their internal behavior in parallel, that's why they get a collection of documents as input. Right now, the new cache support is not thread-safe, so if a module uses it during parallel operations there might be problems.

Allow specifying initial metadata on the command line

Something like --metadata key1=value1;key 2=value 2. This will be helpful in certain scenarios like excluding secrets from a config file.

FTP Module

This module is commonly used at the end of the pipeline, usually after ReadFiles() or ForEachDocument.

The ability to handle common network failure and resume uploading is critical. Consider using this FTP library http://netftp.codeplex.com/documentation.

Create a JavaScript minifier module

Create a Windows service and tray icon

Allow specifying one or more folders to watch. Make the tray icon spin or something when building.

Create a Less CSS module

Should read Less CSS content and transform it to CSS content. Use DotLess.

Implement output web preview

Launch a simple web server. Should point to OutputFolder. Allow specifying port to use.

Cloned repo -> Build fail.

Will look deeper into this tonight; wanted to give quick feedback since you just released.

Add command line argument for recipes

Allow a recipe to be selected from the command line. Should also require the name of the NuGet package where the recipe lives (which will be automatically downloaded). Should also allow the specification of an alternate package source, version, etc. Argument should look something like --recipe package name[@nuget feed]:recipe name.

When a recipe is specified on the command line, and configuration files are ignored. Specifying a recipe and a configuration file on the command line should output an error.

Also need to consider how to configure recipes on the command line. This may impact how recipes are configured in general - for example, only configured in ways the command line can support like string key value pairs.

Implement depth-first traversal for pipelines

See discussion in #25

Create an AutoLink Module

Scans HTML content and automatically generates links given a mapping of keywords to urls. Can't just do a search and replace, have to only link text in the body and in elements where a tags are allowed. Provide a mapping of keywords as a dictionary and also make sure to have an overload that uses a func with a document for custom maps per document. Need to provide an override for content that shouldn't be linked, perhaps a user-defined CSS class. Also need to make sure not to double-link if keyword is already in an a tag.