rocm / rocm-docs-core

ROCm Documentation Python package for ReadTheDocs build standardization

ROCm Documentation Core Utilities

ROCm Docs Core is also distributed as a pip package available from PyPi as rocm-docs-core

Purpose

This repository is comprised of utilities, styling, scripts, and additional HTML content that is common to all ROCm projects' documentation. This greatly aids in maintaining the documentation, as any change to the appearance only needs to be modified in one location.

Usage

Setup

• Install this repository as a Python package using pip

  • From PyPi: pip install rocm-docs-core
  • From GitHub: pip install git+https://github.com/ROCm/rocm-docs-core.git.

• Set rocm_docs_theme as the HTML theme

• Add rocm_docs as an extension

• Optionally, add rocm_docs.doxygen and sphinxcontrib.doxylink as extensions

For an example, see the test conf.py

Legacy Setup

• From the rocm_docs package import the function setup_rocm_docs into conf.py for the ReadTheDocs project.
• Call exactly the following, replacing <PROJECT NAME HERE> with the name of the project.

For an example, see the test legacy conf.py

Documentation

The rocm-docs-core documentation is viewable at https://rocm.docs.amd.com/projects/rocm-docs-core/en/latest/

User Guide

The User Guide describes how users can make use of functionality in rocm-docs-core

It is viewable at https://rocm.docs.amd.com/projects/rocm-docs-core/en/latest/user_guide/user_guide.html

Developer Guide

The Developer Guide provides additional information on the processes in toolchains for rocm-docs-core

It is viewable at https://rocm.docs.amd.com/projects/rocm-docs-core/en/latest/developer_guide/developer_guide.html

Build Documentation Locally

To build the rocm-docs-core documentation locally, run the commands below:

pip install -r requirements.txt
cd docs
python3 -m sphinx -T -E -b html -d _build/doctrees -D language=en . _build/html

rocm-docs-core's Issues

Customize Page titles to include version similar to header text

should show ROCm version

Generated _toc.yml should be in the build directory

Pitchfork project layout

AFAIK it is the intention to follow the Pitchfork project layout. If that is the case, I would like to suggest moving as many of the hidden configuration files in the root of the project into a tools folder, as per § 2.8 of the spec:

The tools/ directory is designated for holding extra scripts and tools related to developing and contributing to the project. For example, turn-key build scripts, linting scripts, code-generation scripts, test scripts, or other tools that may be useful to a project develop.

The contents of this directory should not be relevant to a project consumer.

At the time of writing this touches upon the following files:

• .markdownlint-cli2.yaml
• .pre-commit-config.yaml
• .readthedocs.yaml
• .spellcheck.yaml
• .wordlist.txt

There are more hidden files which aren't needed for users of the project, but some are location-sensitive and can't be placed elsewhere:

• .dockerignore
• .gitignore
• .gitpod.yml

Fix linking

https://rocm.docs.amd.com/projects/rocm-docs-core/en/latest/

GitHub header directs to https://github.com/RadeonOpenCompute/rocm-core instead of https://github.com/RadeonOpenCompute/rocm-docs-core

Format link to main docs

• Less space between ROCm Documentation Home link and Docs page Home link
• Bigger ROCm Documentation Home link

File copy might be needlessly complicated

The file copy in rocm-docs core makes use of the handwritten copy_from_package function:
https://github.com/RadeonOpenCompute/rocm-docs-core/blob/e9c03741abeb0f8043943ce65e16e0d37d6d5de5/src/rocm_docs/__init__.py#L310-L333

This is the same as shutil.copytree outside of the custom error message. Is there some hidden problem with the standard library method or can this be simplified?

More accurate default auto publication date

Article info currently sets default publication date to Python's datetime.date.today

Using the time that the file was last edited would be more accurate.

Add version selector to right side of second level header

This should be a dropdown that mirrors the functionality of the Read the Docs version selector in the bottom left.

Add sphinx-sitemap extension

Add user guide for linking/cross-referencing using intersphinx

https://github.com/RadeonOpenCompute/rocm-docs-core/blob/develop/src/rocm_docs/data/projects.yaml

Unneeded internet access

When building the documentation for hipcc, which uses this package, the system attempts to download inventories (objects.inv) files from the 3 repos listed in the intersphinx_mapping around https://github.com/RadeonOpenCompute/rocm-docs-core/blob/develop/src/rocm_docs/core.py#L91 and generates warnings if they can't be downloaded.

All this appears to do is slow the build down. The generated documentation appears to be the same regardless of the success/failure of the download.

Bump version of rocm-docs-core with every minor change

Change docs build directory to build/docs

To change the output directory for sphinx we should change the argument passed into the doctree pickles -d and <outputdir>

Ref: https://www.sphinx-doc.org/en/master/man/sphinx-build.html

Before:
python3 -m sphinx -T -E -b html -d _build/doctrees -D language=en . _build/html
After:
python3 -m sphinx -T -E -b html -d build/doctrees -D language=en . build/html

Links in headers need updates

The GitHub Link should go to the github page of the current documentation page.

The support page should take you to an issue creation page in the current repository.

Please add code to create this link based on the repository that we are building for.

All the links should open in a new window or tab in the browser.

Document projects.yaml

          > Projects should be defined in projects.yaml and should set which key

they correspond their by setting external_projects_current_project
to this key.

I.e. given this projects.yaml

projects:
  python: https://docs.python.org/3/
  rtd: https://docs.readthedocs.io/en/stable/
  sphinx: https://www.sphinx-doc.org/en/master/
  rocm-docs-core: [https://rocm.docs.amd.com/projects/rocm-docs-core/en/${version}](https://rocm.docs.amd.com/projects/rocm-docs-core/en/$%7Bversion%7D)

conf.py from rocm-docs-core should contain:

external_projects_current_project = "rocm-docs-core"

Adds support for specifying the "development" branch for each project
defined in projects.yaml, this is achieved by setting the
development_branch key for the project I.e.:

projects:
  ...
  project:
    target: <target-url>
    development_branch: master

With this the mapping between projects is changed to the following

• Development branches map between each-other,
  i.e. links on the "develop" branch of project A point to the "master"
  branch of project "B" and vice-versa if A has set its
  "development branch" to develop and B sets it to "master".

• Symbolic versions "latest" and "stable" map to themselves in other
  projects

• Any other branch maps to "latest"

---

This commit message is very helpful for using these settings. I wonder if we should add a user guide in the docs pages.

Originally posted by @samjwu in #141 (review)

Reduce number of latest versions to update in header

in 2 locations

• theme.conf
• project.py

reduce to 1 location

Update banner message based on branch

On the newest tagged version, there should be no banner

On develop/master, the banner should state that the page is unreleased or a future version

For others, the banner should state that the page is an old version (i.e.: not latest)

v0.18.3 breaks TOC generation

This commit doesn't only add projects, but also renames them (seemingly for cosmetic reasons). This breaks TOC generation for all docs even in retrospect. The point of this file is to give stable names to links which may potentially change overtime.

Changing migraphx to amdmigraphx and composable-kernel to composable_kernel breaks all working branches, but will also break all of the backported and non-latest branched versions of the docs.

Please revert the aesthetic changes, which isn't worth the effort of having to update and retag all our pre-existing docs.

PDF Name fixes need to be corrected on generated documentation

Declare parallel reads

Building a repository with the -j (parallel build) option that uses rocm-docs-core gives the following warning:

WARNING: the rocm_docs extension does not declare if it is safe for parallel reading, assuming it isn't - please ask the extension author to check and make it explicit
WARNING: doing serial read

Add suggestions Link on header

suggestions link should lead to page on Main page with request to submit new GitHub issue or email documentation.

Cookie settings link color coding broken

Please take a look at the cookie setting link and restore the look.

Auto read time

Add auto "read time" based on doc length in article info.

eg: 1 min per 100 char

Set HTML title to '[page] - AMD ROCm Documentation'

Documentation Version Number in CMakeLists.txt

Get the version from CMakeLists.txt
Use rocm_setup_version string by default
If it doesn't exist, do custom regex in conf.py

example: rocBLAS
rocm_setup_version( VERSION ${VERSION_STRING} )

Add release candidate to intersphinx links (projects.yml)

syntax is release/rocm-rel-5.6 or rocm-rel-5.6 or roc-5.6.x

Fix header dropdown on narrower screens

Add way to differentiate Windows and Linux documentation

Some documentation is meant for Linux or Windows users only.

To convey that clearly, a way is needed to indicate that.

Consider Windows and Linux icons on pages or selectively removing pages using a toggle.

Need a common 404 page for docs

Method 1: Copy over a 404.md via rocm-docs-core to docs directory for each repo before Sphinx builds docs

Method 2: Point all to the ROCm 404 page

Method 3: Create 404.md for each repo

Documentation Standardization

Apply the following changes to project documentation for ReadtheDocs:

• add version number to documentation left navigation bar and page title
• add an "About" section with a license page
• enable htmlzip, pdf, epub formats when publishing on Read the Docs
• set pdf title, author, copyright, and version
• rename .sphinx/.doxygen to sphinx/doxygen
• remove docBin from URL
• update rocm-docs-core dependency
• update dependabot config

List

• GPUOpen-ProfessionalCompute-Libraries MIVisionX
• GPUOpen-ProfessionalCompute-Libraries rpp
• ROCm-Developer-Tools HIP
• ROCm-Developer-Tools HIP-VS
• ROCm-Developer-Tools HIPCC
• ROCm-Developer-Tools HIPIFY
• ROCm-Developer-Tools ROCdbgapi-docs
• ROCm-Developer-Tools ROCgdb-docs
• ROCm-Developer-Tools ROCmValidationSuite
• ROCm-Developer-Tools rocprofiler-docs
• ROCm-Developer-Tools roctracer-docs
• ROCmSoftwarePlatform AMDMIGraphX
• ROCmSoftwarePlatform MIOpen
• ROCmSoftwarePlatform TransferBench
• ROCmSoftwarePlatform composable_kernel
• ROCmSoftwarePlatform hip-python main
• ROCmSoftwarePlatform hipBLAS
• ROCmSoftwarePlatform hipBLASLt
• ROCmSoftwarePlatform hipCUB
• ROCmSoftwarePlatform hipFFT
• ROCmSoftwarePlatform hipRAND
• ROCmSoftwarePlatform hipSOLVER
• ROCmSoftwarePlatform hipSPARSE
• ROCmSoftwarePlatform hipSPARSELt
• ROCmSoftwarePlatform hipTensor
• ROCmSoftwarePlatform hipfort
• ROCmSoftwarePlatform rccl
• ROCmSoftwarePlatform rocAL
• ROCmSoftwarePlatform rocALUTION
• ROCmSoftwarePlatform rocBLAS
• ROCmSoftwarePlatform rocFFT
• ROCmSoftwarePlatform rocPRIM
• ROCmSoftwarePlatform rocRAND
• ROCmSoftwarePlatform rocSOLVER
• ROCmSoftwarePlatform rocSPARSE
• ROCmSoftwarePlatform rocThrust
• ROCmSoftwarePlatform rocWMMA
• RadeonOpenCompute ROCR-Runtime
• RadeonOpenCompute ROCm
• RadeonOpenCompute amdsmi
• RadeonOpenCompute rdc
• RadeonOpenCompute rocm-cmake
• RadeonOpenCompute rocm-docs-core
• RadeonOpenCompute rocm_smi_lib

Add link to AMD Lab Notes in header

https://gpuopen.com/learn/amd-lab-notes/amd-lab-notes-readme/ link should be added after infinity hub.

formatting, static type checking and pep8 compliance.

Enable black formatter, pylint and pycode-style via github action

Change link colors

Use #2899FF for dark backgrounds, and #0051C6 for light

Create test cases

There should be test cases to validate the functionality of rocm-docs-core for both legacy and theme users (aptly demonstrated by the confusion around #88).

Add reference to markdown lint rules in user guide

ref: https://github.com/DavidAnson/markdownlint#rules–aliases

guide: https://rocm.docs.amd.com/projects/rocm-docs-core/en/latest/user_guide/linting.html

also mention how it can be installed and run locally

Add version numbers to generated documentation

The built documentation should have the version number somewhere

github link is broken on header

Set up separate source for components for updates

create a separate source for components that are frequently updated
rocm-docs-core can reference these components instead of hard-coding them

Goal: make updating components that need frequent changes (eg: announcements) easier in that it won't require bumping versions and updating dependencies for all libraries for small changes

Possibilities:

• separate repo
• gist
• separate file in main or develop branch

Documentation Index should be turned off for the main ROCm repository.

Allow configuration for documentation for non ROCm sites

Custom linting rule for links in markdown

Make custom markdown rule to disallow extensions in markdown links

Example:

[words](filename) is valid

[words](filename.md) is not valid

Refs:

https://github.com/DavidAnson/markdownlint/blob/main/doc/CustomRules.md

https://www.npmjs.com/package/markdownlint-rule-search-replace

Version substition

Problem case

Currently in main ROCm documentation, versions are kept up to date manually.
Updating this is error prone (i.e. forgetting to update a string).
Having a mechanism to do this programmatically would help greatly.

Myst already has substitutions, but this does not work within code blocks, which is mainly used for showing install instructions.

Possible solutions

• Add a simple string replace pre-processing step.

• There is a sphinx extension that might do the job, however this does not work with Myst. There is a PR that resolves that but requires additional fix-ups to get merged: adamtheturtle/sphinx-substitution-extensions#235

• Add a Jinja (or other templating engine) pre-processing step that processes *.md.jinja or *.jinja.md-files first before cotinuining with normal Myst/Sphinx steps.

  • Myst already has some form of Jinja support with substitutions, so this might conflict.
  • Depending on your point of view, this might be useful/overkill.

Read Time should be rounded to the nearest minute.

Do not include .0

Add user guide for linting

PR #207 did not include full user documentation

Tarball or zip based projects downloads should still be able to build docs.

Input files with LaTeX formulas are changed during build

In the hipfort docs build, equations seem to be causing some troubles with the copy.

lib/hipfort/hipfort_hipblas.f:

  !>  \brief SOLVER API
  !> 
  !>     \details
  !>     getrs solves a system of n linear equations on n variables in its factorized form.
  !> 
  !>     It solves one of the following systems, depending on the value of trans:
  !> 
  !>     \f[
  !>         \begin{array}{cl}
  !>         A X = B & \: \text{not transposed,}\\
  !>         A^T X = B & \: \text{transposed, or}\\
  !>         A^H X = B & \: \text{conjugate transposed.}
  !>         \end{array}
  !>     \f]

docs/doxygen/input/hipfort_hipblas.f:

  !>  \brief SOLVER API
  !> 
  !>     \details
  !>     getrs solves a system of n linear equations on n variables in its factorized form.
  !> 
  !>     It solves one of the following systems, depending on the value of trans:
  !> 
  !>     \f[
  !>         \begin{array}{cl}
  !>         A X = B & \: \text{not transposed,}\  !>         A^T X = B & \: \text{transposed, or}\  !>         A^H X = B & \: \text{conjugate transposed.}


  !>         \end{array}
  !>     \f]

rocm-docs-core should be a sphinx extension

As noted in #32 rocm-docs-core should provide all of its functionality as a sphinx extension.

The sphinx API seems to provide all the features that rocm-docs-core needs (i.e. requiring other extensions, modifying settings, installing files.)

This would:

• clean up the consumer interface (no more copying variables into the global scope)
• clean up rocm-docs-core internals (it is currently partly a plugin and partly not, some code is duplicated so that calls from clients can happen in any order)
• would allow for more modularization making parts of the project useful outside of rocm documentation (i.e. running doxygen and doxysphinx automatically)

Add article info on each page

https://sphinx-design.readthedocs.io/en/latest/additional.html

Article info author can be the supported OS, Windows, Linux, Windows and Linux.

Use font awesome pro for win/lin icons

Larger base font size

Preferred size is 14-16px (0.875rem – 1rem)

No Index to prevent search engine crawling

<meta name="robots" content="noindex,follow" />