stjudecloud / wdldoc

Create WDL documentation using Markdown.

wdldoc

Convert WDL documentation to Markdown for rendering.
Request Feature · Report Bug · ⭐ Consider starring the repo! ⭐

📚 Getting Started

For an example of what the results can look like, check out the GitHub Pages for the St Jude Cloud Workflows repo! The documentation is automatically built for each release using wdldoc.

Installation

wdldoc is only available for Python 3.8 or higher.

Suggested install method:

conda create -n wdldoc python=3.8
conda activate wdldoc
pip install wdldoc

Usage

wdldoc is designed to be simple, and require as little work as possible. Once installed, simply call wdldoc . at the root of your WDL project, and Markdown files will be generated in the ./documentation directory for each WDL file found. There are tasks/ and workflows/ subdirectories, with documentation for WDL workflow files in workflows/, and documentation for WDL task files in tasks/.

Any valid WDL file will have the inputs, outputs, and meta information individually documented for all its tasks and workflows. There's no need to conform to any standards we dictate; if it runs, we'll document it.

Any strings found in meta fields will be treated as Markdown, so feel free to add custom bolding, italicizing, code snippets, etc.

If there's any information you want to include for a file that doesn't fit into a meta field of one of it's tasks or workflows, you can include a header section of your WDL file, and we'll convert it to Markdown and prepend it to the documentation. This is a good place to document the uses for the file and any licensing information. Simply start a line with ## followed by a space, and the rest of the line will be parsed as Markdown. There's no limit to the number of header lines. It's good practice to break up the header into sections using Markdown titles.

usage: wdldoc [-h] [-o OUTPUT_DIRECTORY] [-d DESCRIPTION] [-c CHOICES] [-v] [--debug] sources [sources ...]

Generate clean WDL documentation from source.

positional arguments:
  sources               Top level directories to search for WDL files, or the WDL files themselves.

optional arguments:
  -h, --help            show this help message and exit
  -o OUTPUT_DIRECTORY, --output_directory OUTPUT_DIRECTORY
                        Directory to store markdown files. Default is `./documentation`
  -d DESCRIPTION, --description DESCRIPTION
                        If parameter meta fields use a JSON object, the key for the field containing the input description. Default is 'help'. Ignored if only strings are used.
  -c CHOICES, --choices CHOICES
                        If parameter meta fields use a JSON object, the key for the field containing the input choices. Default is 'choices'. Ignored if only strings are used.
  -v, --verbose         Sets the log level to INFO.
  --debug               Sets the log level to DEBUG.

Either directories or individual files can be supplied. When directories are supplied, wdldoc will recursively search the input directories searching for all .wdl files, and generate documentation for them.

WDL parameter_meta info can be anything that conforms to the WDL spec, but we recommend one of two formats. The first is simply input_name: "descriptive string". The other is a JSON object containing a description key with a string value and optionally a choices key with a list of options. The value of the "description" and "choices" keys can be specified with the --description and --choices arguments. Below is an example of both formats in one parameter meta block.

parameter_meta {
    in_bams: {
        help: "Provide bams to run for comparison"
    }
    tissue_type: {
        help: "Provide the tissue type to compare against",
        choices: ['blood', 'brain', 'solid']
    }
    output_filename: "Name for the output HTML t-SNE plot"
}

🖥️ Development

If you are interested in contributing to the code, please first review our CONTRIBUTING.md document. To bootstrap a development environment, please use the following commands.

# Clone the repository
git clone [email protected]:stjudecloud/wdldoc.git
cd wdldoc

# Install the project using poetry
poetry install

# Ensure pre-commit is installed to automatically format
# code using `black`.
brew install pre-commit
pre-commit install
pre-commit install --hook-type commit-msg

📝 License

Copyright © 2020 St. Jude Cloud Team.

This project is MIT licensed.

wdldoc's Issues

The automated release is failing 🚨

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you could benefit from your bug fixes and new features.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can resolve this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit to the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here is some links that can help you:

• Usage documentation
• Frequently Asked Questions
• Support channels

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.

---

Missing package.json file.

A package.json file at the root of your project is required to release on npm.

Please follow the npm guideline to create a valid package.json file.

---

Good luck with your project ✨

Your semantic-release bot 📦🚀

Request: support for struct documentation

Hi,

Would it be possible to add support to auto-document structs? We have workflows/tasks that take custom structs as inputs, but as of now, these inputs are somewhat undocumented due to the missing struct support.

Thanks
M

Support for WDL development version

Hi,

Is there any support planned for WDL v1.1 and above?
I've got some WDL's in written in version development for which wdldoc reports validation error, but pass in womtool validate

Thanks
M

Support for check_quant=False when loading wdls

A lot of our workflows have optional inputs and using wdldoc fails to create documentation for these workflows with an import error. Would it be possible to add an option to pass check_quant=False to wdl.load?

example error
[E 230126 16:47:23 wdldoc:67] ImportError: Failed to import cds_prediction.wdl

When i try directly with
wdl.load("cds_prediction.wdl")
I get a try error
WDL.Error.StaticTypeMismatch: Expected String instead of String? -- to coerce T? X into T, try select_first([X,defaultValue]) or select_first([X]) (which might fail at runtime); to coerce Array[T?] X into Array[T], try select_all(X)

If I add check_quant=False the wdl loads w/o a problem

>>> wdl.load("cds_prediction.wdl",check_quant=False)
<WDL.Tree.Document object at 0x155552dc3a60>