return42 / linuxdoc

Sphinx-doc extensions for sophisticated C developer

LinuxDoc

The LinuxDoc library contains Sphinx-doc extensions and command line tools to extract documentation from C/C++ source file comments. Even if this project started in context of the Linux-Kernel documentation, you can use these extensions in common Sphinx-doc projects.

Install

Install LinuxDoc using pip:

pip install --user -U linuxdoc

Links

• Documentation: https://return42.github.io/linuxdoc
• Releases: https://pypi.org/project/linuxdoc/
• Code: https://github.com/return42/linuxdoc
• Issue tracker: https://github.com/return42/linuxdoc/issues

package:	linuxdoc (20231020)
copyright:	2023 Markus Heiser
e-mail:	[email protected]
license:	AGPLv3+

linuxdoc's Issues

Lets have an eye on patches of the origin kernel-doc script from the kernel sources

This is the history of the origin kernel-doc script (Perl) we should have an eye on:

• https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/scripts/kernel-doc?qt=grep&q=

• https://salsa.debian.org/guest/kernel-doc/-/commits/rework/scripts/kernel-doc

possible candidates for a takeover:

• kernel-doc: better handle '::' sequences which was improved by kernel-doc: better handle '::' sequences and docs: kernel-doc: properly recognize parameter lines with colons
• kernel-doc: improve parsing for kernel-doc comments syntax
• scripts: kernel-doc: reduce repeated regex expressions into variables e.g. to optimize FUNC_PROTOTYPES ..

linuxdoc/linuxdoc/kernel_doc.py

Lines 215 to 234 in 365757c

	FUNC_PROTOTYPES = [
	# RE(r"^(\w+)\s+\(\*([a-zA-Z0-9_]+)\)\s*\(([^\(]*)\)") # match: void (*foo) (int bar);
	RE(r"^()([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)")
	, RE(r"^(\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)")
	, RE(r"^(\w+\s*\*+)\s*([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)")
	, RE(r"^(\w+\s+\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)")
	, RE(r"^(\w+\s+\w+\s*\*+)\s*([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)")
	, RE(r"^(\w+\s+\w+\s+\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)")
	, RE(r"^(\w+\s+\w+\s+\w+\s*\*+)\s*([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)")
	, RE(r"^()([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)")
	, RE(r"^(\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)")
	, RE(r"^(\w+\s*\*+)\s*([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)")
	, RE(r"^(\w+\s+\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)")
	, RE(r"^(\w+\s+\w+\s*\*+)\s*([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)")
	, RE(r"^(\w+\s+\w+\s+\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)")
	, RE(r"^(\w+\s+\w+\s+\w+\s*\*+)\s*([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)")
	, RE(r"^(\w+\s+\w+\s+\w+\s+\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)")
	, RE(r"^(\w+\s+\w+\s+\w+\s+\w+\s*\*+)\s*([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)")
	, RE(r"^(\w+\s+\w+\s*\*\s*\w+\s*\*+\s*)\s*([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)")
	]

question about doc in python source code

In linuxdoc's python source code. There are many doc comment. Could you tell me what format standard they were followed?

AttributeError: 'FlatTable' object has no attribute 'set_table_width'

Hello!

I use flat-table and have the build error:

Exception occurred:
  File "/home/ec2-user/.local/lib/python3.7/site-packages/linuxdoc/rstFlatTable.py", line 113, in run
      self.set_table_width(tableNode)
 AttributeError: 'FlatTable' object has no attribute 'set_table_width'

Could you help me, please?

Sphinx version: 5.3.0
Python version: 3.7.16
Linuxdoc version: 3.0

Expected identifier in nested name

Hi,

I upgraded from an old sphinx version (<3.0) to sphinx 5.3, by the way I upgraded linuxdoc to the latest version available on PIP (20221025).

But now I got an error while parsing parsing a simple macro, sphinx output:

/workspace/doc/minimalist/source.c:3: WARNING: Error in declarator
If declarator-id with parameters (e.g., 'void f(int arg)'):
  Invalid C declaration: Expected identifier in nested name. [error at 3]
    FOO()

Here is my conf.py

# -*- coding: utf-8 -*-
#
# Configuration file for the Sphinx documentation builder.
#
# This file does only contain a selection of the most common options. For a
# full list see the documentation:
# http://www.sphinx-doc.org/en/master/config

project = 'test'
author = 'test'

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
    'linuxdoc.rstKernelDoc',
    'linuxdoc.cdomain',
]

index.rst:

.. kernel-doc:: source.c

source.c

/**
 * FOO - A macro
 *
 * macro is set tro 1
 */
#define FOO 1

I tried to replace the c:function by a c:macro and it fix the issue. So I'm wondering if sphinx didn't broke something with c:function.

Having a function with the same name as an enum or a struct makes a collision

First of all, thank you for this project, it makes using the kernel-doc format much simpler and the output format is nicer than with the kernel-doc script.

Now my problem:

If I define an enum (or a struct) and a fonction with the same name:

/**
 * enum foo - Foo enum.
 */
enum foo
{
};

/**
 * foo() - Foo function.
 *
 * Can I reference foo() and &enum foo?
 */
void foo(void);

It produce the warnings:

.../foo.h:6: [kernel-doc WARN] : Enum value '' not described in enum 'foo'
.../foo.h:13: [kernel-doc ERROR] : name 'foo' used several times
.../foo.h:13: [kernel-doc ERROR] : name 'foo' used several times

.../foo.h:10: WARNING: Duplicate explicit target name: "foo".

And of course, the references do not work.

Can this be fixed or is it something sphinx is doing?

I use sphinx 2.4.4 and linuxdoc 20210110.

Thanks.

Could not import extension linuxdoc.cdomain

If you get a Could not import extension linuxdoc.cdomain .. update your linuxdoc installation. The bug is fixed by commit 48f09de.

Support for class and name options

It would be nice for flat-tables to allow the common options class and name, like the standard list-table

Empty cell in flat-table results in exception

A recent commit seems to have broken flat-table's ability to describe an empty cell

The following:

FlatTable bug
=============

.. flat-table:: 
    :header-rows: 1
    
    *   - [1,1]
        - [2,1]
    
    *   - [1,2]
        - 

Results in:

Traceback (most recent call last):
  File "<hidden>/site-packages/sphinx/cmd/build.py", line 304, in build_main
    app.build(args.force_all, filenames)
  File "<hidden>/site-packages/sphinx/application.py", line 341, in build
    self.builder.build_update()
  File "<hidden>/site-packages/sphinx/builders/__init__.py", line 347, in build_update
    len(to_build))
  File "<hidden>/site-packages/sphinx/builders/__init__.py", line 360, in build
    updated_docnames = set(self.read())
  File "<hidden>/site-packages/sphinx/builders/__init__.py", line 468, in read
    self._read_serial(docnames)
  File "<hidden>/site-packages/sphinx/builders/__init__.py", line 490, in _read_serial
    self.read_doc(docname)
  File "<hidden>/site-packages/sphinx/builders/__init__.py", line 534, in read_doc
    doctree = read_doc(self.app, self.env, self.env.doc2path(docname))
  File "<hidden>/site-packages/sphinx/io.py", line 318, in read_doc
    pub.publish()
  File "<hidden>/site-packages/docutils/core.py", line 217, in publish
    self.settings)
  File "<hidden>/site-packages/docutils/readers/__init__.py", line 72, in read
    self.parse()
  File "<hidden>/site-packages/docutils/readers/__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "<hidden>/site-packages/sphinx/parsers.py", line 88, in parse
    self.statemachine.run(inputstring, document, inliner=self.inliner)
  File "<hidden>/site-packages/docutils/parsers/rst/states.py", line 171, in run
    input_source=document['source'])
  File "<hidden>/site-packages/docutils/statemachine.py", line 239, in run
    context, state, transitions)
  File "<hidden>/site-packages/docutils/statemachine.py", line 460, in check_line
    return method(match, context, next_state)
  File "<hidden>/site-packages/docutils/parsers/rst/states.py", line 2753, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "<hidden>/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "<hidden>/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
    node=section_node, match_titles=True)
  File "<hidden>/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
    node=node, match_titles=match_titles)
  File "<hidden>/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "<hidden>/site-packages/docutils/statemachine.py", line 239, in run
    context, state, transitions)
  File "<hidden>/site-packages/docutils/statemachine.py", line 460, in check_line
    return method(match, context, next_state)
  File "<hidden>/site-packages/docutils/parsers/rst/states.py", line 2753, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "<hidden>/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "<hidden>/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
    node=section_node, match_titles=True)
  File "<hidden>/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
    node=node, match_titles=match_titles)
  File "<hidden>/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "<hidden>/site-packages/docutils/statemachine.py", line 239, in run
    context, state, transitions)
  File "<hidden>/site-packages/docutils/statemachine.py", line 460, in check_line
    return method(match, context, next_state)
  File "<hidden>/site-packages/docutils/parsers/rst/states.py", line 2326, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "<hidden>/site-packages/docutils/parsers/rst/states.py", line 2338, in explicit_construct
    return method(self, expmatch)
  File "<hidden>/site-packages/docutils/parsers/rst/states.py", line 2081, in directive
    directive_class, match, type_name, option_presets)
  File "<hidden>/site-packages/docutils/parsers/rst/states.py", line 2130, in run_directive
    result = directive_instance.run()
  File "<hidden>/site-packages/linuxdoc/rstFlatTable.py", line 123, in run
    tableBuilder.parseFlatTableNode(node)
  File "<hidden>/site-packages/linuxdoc/rstFlatTable.py", line 220, in parseFlatTableNode
    row = self.parseRowItem(rowItem, rowNum)
  File "<hidden>/site-packages/linuxdoc/rstFlatTable.py", line 339, in parseRowItem
    cspan, rspan, cellElements = self.parseCellItem(cellItem)
  File "<hidden>/site-packages/linuxdoc/rstFlatTable.py", line 351, in parseCellItem
    for elem in cellItem[0]:
  File "<hidden>/site-packages/docutils/nodes.py", line 569, in __getitem__
    return self.children[key]
IndexError: list index out of range

Exception occurred:
  File "<hidden>/site-packages/docutils/nodes.py", line 569, in __getitem__
    return self.children[key]
IndexError: list index out of range

How to give path to srctree?

Hello,

the only way I found to give the path to srctree is to export the srctree variable before calling sphinx-build.

Is there a way to do it from the conf.py file?

Thanks.

Edit: s/srcdir/srctree/g

False positive WARNING from kernel-doc-man builder

The kenerl-doc-man builder produces false positive warnings when using the :internal: option of the kernel-doc directive:

docs/linuxdoc-howto/test_internals.c:24: WARNING: Duplicate explicit target name: "test_internals_a.bar".
docs/linuxdoc-howto/test_internals.c:24: WARNING: Duplicate C declaration, also defined at linuxdoc-howto/kernel-doc-tests:24.
Declaration is '.. c:function:: int bar(int a,  ...)'.
docs/linuxdoc-howto/test_internals.c:27: WARNING: Duplicate explicit target name: "test_internals_a.bar.description".
docs/linuxdoc-howto/test_internals.c:30: WARNING: Duplicate explicit target name: "test_internals_a.bar.example".
docs/linuxdoc-howto/test_internals.c:33: WARNING: Duplicate explicit target name: "test_internals_a.bar.return".
docs/linuxdoc-howto/test_internals.c:43: WARNING: Duplicate explicit target name: "test_internals_a.internal_function".
docs/linuxdoc-howto/test_internals.c:43: WARNING: Duplicate C declaration, also defined at linuxdoc-howto/kernel-doc-tests:43.
Declaration is '.. c:function:: int internal_function( void)'.
docs/linuxdoc-howto/test_internals.c:45: WARNING: Duplicate explicit target name: "test_internals_a.internal_function.context".
docs/linuxdoc-howto/test_internals.c:47: WARNING: Duplicate explicit target name: "test_internals_a.internal_function.return".
docs/linuxdoc-howto/test_internals.c:4: WARNING: Duplicate explicit target name: "test_internals_b.foo".
docs/linuxdoc-howto/test_internals.c:4: WARNING: Duplicate C declaration, also defined at linuxdoc-howto/kernel-doc-tests:4.
Declaration is '.. c:function:: int foo(int a,  ...)'.
docs/linuxdoc-howto/test_internals.c:7: WARNING: Duplicate explicit target name: "test_internals_b.foo.description".
docs/linuxdoc-howto/test_internals.c:10: WARNING: Duplicate explicit target name: "test_internals_b.foo.example".
docs/linuxdoc-howto/test_internals.c:13: WARNING: Duplicate explicit target name: "test_internals_b.foo.return".
docs/linuxdoc-howto/test_internals.c:43: WARNING: Duplicate explicit target name: "test_internals_b.internal_function".
docs/linuxdoc-howto/test_internals.c:43: WARNING: Duplicate C declaration, also defined at linuxdoc-howto/kernel-doc-tests:43.
Declaration is '.. c:function:: int internal_function( void)'.
docs/linuxdoc-howto/test_internals.c:45: WARNING: Duplicate explicit target name: "test_internals_b.internal_function.context".
docs/linuxdoc-howto/test_internals.c:47: WARNING: Duplicate explicit target name: "test_internals_b.internal_function.return".
docs/linuxdoc-howto/kernel-doc-tests.rst:355: WARNING: using option :export: but there are no exported symbols

Installation of linuxdoc breaks linting local tests folder

Hi,

I stumbled upon a very strange behavior of pylint after installing this package.
We have got a project, that uses sphinx to create documentation and want to use the flat-table directive from the linuxdoc package.
I created a PR adding the library as a dependency. The only change is that I have added linuxdoc to the pyproject.toml.
After that, pylint started reporting the following error (when trying to lint our tests repository, containing tests for our project):

After some research, I noticed that after installing linuxdoc, a folder "tests" is located under "{venv}/Lib/site-packages/tests".
Removing this folder solves the issue.
The behavior could be reproduced both on windows and on linux.

I am not an expert when it comes to python packaging, so I don't know if I'm missing something, but my assumption would be, that there should not be a "tests" folder inside the site-packages directory.

BR,
Valentin

Struct definition and field description not showing when using typedefs

Hello,

I am encountering some issues when trying to document my structs when making using of typedefs during the definition.
Let's say I have this kind of struct definition:

/**
 * typedef my_struct_t - short description
 * @foo: foo
 * @bar: bar
 */
typedef struct my_struct {
    int foo;
    float bar;
} my_struct_t;

And in my doc.rst I have the following reST directive:

.. kernel-doc:: my_header.h
    :symbols: my_struct_t

When I try this I am not getting any error and the type appears in the documentation but I am not able to display as well the structure content and the description for each field. The same issue could be applied to enum and all other keywords used with typedefs.
Do you have any kind of solution to be able to display as well the structure and the field descriptions?

Thank you a lot,
Ayoub

N.B.:
I'm currently using sphinx==5.2.3

New release?

I'd like to use the class option introduced in #12 in a project that doesn't have any bleeding edge dependencies.

I also think that the documentation should mention that class and name are only supported since version X. (I just spend way too much time trying to figure out why class wasn't working).

Unknown node rowSpan

Failed to build sphinx html with FlatTable using linuxdoc
raise NotImplementedError('Unknown node: ' + node.__class__.__name__)
NotImplementedError: Unknown node: rowSpan

#: ../../introduction.md:1
msgid "{rspan}1 TEST"
msgstr "{rspan}1 TEST"

The command that I use
python -m sphinx -T -E -b html -d _build/doctrees -D language=ru . _build/html

Update:

I used the same example as in the documentation, but the same problem

index.rst


Test
----

.. flat-table:: table title
   :widths: 2 1 1 3

   * - head col 1
     - head col 2
     - head col 3
     - head col 4

   * - column 1
     - field 1.1
     - field 1.2 with autospan

   * - column 2
     - field 2.1
     - :rspan:`1` :cspan:`1` field 2.2 - 3.3

   * .. _`last row`:

     - column 3

and locales/ru/LC_MESSAGES/index.po

#: ../../test/index.rst:18
msgid ":rspan:`1` :cspan:`1` field 2.2 - 3.3"
msgstr ":rspan:`1` :cspan:`1` test 2.2 - 3.3"

Incorrect style in cells when merging stub columns

Hi!

1. I try to merge stub columns:

.. flat-table:: flat-table example
   :header-rows: 1
   :stub-columns: 1
   :widths: 1 1 1 1 2
   :class: my-class
   :name: my-flat-table

   * - 
     - head 2.1
     - head 2.2
     - head 2.3
     - head 2.4

   * - :rspan:`2` row 1-3
     - cell 1.1
     - cell 1.2
     - cell 1.3
     - cell 1.4

   * - cell 2.1
     - cell 2.2
     - cell 2.3
     - cell 2.4

   * - cell 3.1
     - cell 3.2
     - cell 3.3
     - cell 3.4

   * - row 4
     - cell 4.1
     - cell 4.2
     - cell 4.3
     - cell 4.4

2. Cell 2.1 and Cell 3.1 has incorrect style:
   

Please include a license file in the sources

Hello!

Similar to return42/fspath#1, could we please have a license file added here too?

Thanks very much,

Python 3.12: `imp` has been removed

imp module is removed in Python 3.12

linuxdoc/setup.py

Line 18 in 5bd1bfd

PKG = imp.load_source('__pkginfo__', ospj(SRC, '__pkginfo__.py'))

The suggested replacement is to use importlib

Related:

• python/cpython#98040
• return42/fspath#3

Please publish to PyPi

Currently, linuxdoc documentation recommends installation of the extension via the GitHub git path. Any reason why you don't publish it as a package to PyPi?

I think you have an excellent set of plugins. Publishing to the Python package index would be a great way to share this work!