mpark / wg21 Goto Github PK

I updated the submodule for wg21 today and papers with captioned tony-tables fail with:

Error producing PDF.
! Use of \LT@array doesn't match its definition.
\new@ifnextchar ...served@d = #1\def \reserved@a {
                                                  #2}\def \reserved@b {#3}\f...
l.499 (simplified for clarity)}

When collecting and forming the references at the end of the document, the pdf rendering cannot handle an empty set of references, while the html rendering has no trouble with such cases. A typical error that is reported when trying to generate a .pdf looks like:

% make RegularNullablePointer.pdf. // <-- command line

pandoc RegularNullablePointer.md -o generated/RegularNullablePointer.pdf -d ./data/defaults.yaml --bibliography ./data/csl.json
Error producing PDF.
! LaTeX Error: Command \CSLBlock undefined.

See the LaTeX manual or LaTeX Companion for explanation.
Type H for immediate help.
...

l.98 \renewcommand{\CSLBlock}

make: *** [generated/RegularNullablePointer.pdf] Error 43

I came across how-i-format-my-cpp-papers and wanted to ask if you were aware of asciidoc and other alternatives to markdown?

There is no option to comment on your blog. I thought it better to ask here rather than email you directly as it could otherwise be asked a few times.

This is not a criticism or defect report and you are of course free to use whatever you want.
You might interpret this as a feature request for a companion blog entry "why Markdown is for me?" which you are free to reject.

asciidoc, if you don't know of it, is similar to markdown but much less limiting. You can write entire books in it (it can be mapped to docbook).
I've found it a very good fit for small specifications and where I want to generate both html and PDF.

There are other alternatives as well and its quite a deep well to explore but I think its interesting when someone settles on one to understand their reasons. I've observed people often start with pure Markdown and discover the alternatives only when they get fed up with its limitations.
Coming from LaTeX the limitations must have been more visible to you at the outset which should make for an interesting perspective.

While generating a PDF with citations on the latest Pandoc the following error is reported:

Error producing PDF.
! LaTeX Error: Environment cslreferences undefined.

See the LaTeX manual or LaTeX Companion for explanation.
Type  H <return>  for immediate help.
 ...                                              
                                                  
l.384 \begin{cslreferences}

make: *** [../wg21/Makefile:6: generated/2007R0_std_from_chars_should_work_with_std_string_view.pdf] Error 43

I don't get my @-surrounded emphasis to work, instead it just writes the @thing@ to the pdf.

Same problem for html.

Same problem with both __ and ** use .

Here is a sample code block that doesn't work for me, the intent is to boldface the word typename, but instead it just outputs the @** and **@ sequences as is:

class MyClass {
    operator int() { return 0; }
    operator double() { return 1; }
};

auto conv = &MyClass::operator @**typename**@;

MyClass c;

int a = (c.*conv)();
double a = (c.*conv)();

I find it extremely tedious to click on a reference like [PnnnnR0] and get taken to the references section at the end of the paper, where I have to manually scan through the references to find the one I clicked on (because the browser doesn't put it at the first line because we're at the end of the doc), and then click again. I don't care about the nicely formatted references section that makes it look like a nice academic paper. I just want to go to PnnnnR0 and read the flipping paper!

Would it be possible to add a little "link" icon (e.g. U+1F517) after a reference, so that it appears as [PnnnnR0]🔗 and I can click the icon to go straight to the paper, not to the References section?

Personally, I'd just make the reference in square brackets go directly to the paper in the first place ... does anybody read the References? Maybe that info (title, author, date etc) could be presented as a mouseover popup using the title attribute of the <a> element to save jumping to the end of the paper. Most of the time I just want to read the actual paper, so I want to open it in a new browser tab ASAP. I don't want to go to the references at the end then jump back to where I was reading. That workflow dates from papers printed on actual paper, and is just tedious in hypertext.

Instead of what you write in the instructions you have to write:

make generated/<paper>.pdf

This got me stumped for quite some time. Please update the instructions in the README.md file, or better, make it work without the generated/ prefix.

The PDF output does not show add/rm as underline / strikethrough. This is an issue for CWG. #18 makes me prefer PDF, were it not for the lack of underline / strikethrough...

Hi, I get several errors from wg21.py wrt to passing a NoneType to os.fspath()

any ideas how to fix?

I am a complete python newbie....

\pnum{}s are swallowed by HTML output. I use HTML output as CWG really wants to have strikethrough/underline, which doesn't work in PDF, see #19 . But not having paragraph numbers will make CWG at least as unhappy... Whatever is easier to fix (also taking into account #18 - maybe adding underline / strikethrough to PDF is simpler after all?)

Hi there :)

Apologies for my overly negative title, but I have great difficulty getting this to "just work".
I'm on Ubuntu 22, dependencies installed (python3, pip3, pandoc, texlive-full, etc.), and a myriad of errors.
In no particular order:

• had to go to wg21/deps/python/pyvenv.cfg and change the variable include-system-site-packages to true, otherwise it couldn't find the installed Python packages, e.g. requests.
• It complained multiple times about "error reading bibliography file wg21/data/csl.json", in which case I had to manually delete that file before I could run the Makefile again.
• After fixing a lot of problems that should seem unrelated / I don't care, I get an error with a Latex command "\CSBlock" undefined.

There are (asides from this one) 7 open issues unattended.

It seems I must resort to raw Latex, because at least that works :/
This tool looks really cool otherwise, I'm really hoping these issues get fixed / documented.

Kind regards.

If I have markdown like:

Some clause

[1]{.pnum} Some numbered thing.

Running that through the filters here produces:

<p>Some clause</p>
<p><div class="marginalizedparent"><a class="marginalized">1</a></div> Some numbered thing.</p>

But as I've learned today, you're not allowed to put a <div> in a <p>. This ends up doing all sorts of nonsense when I try to add custom formatting.

On Debian I had to install the following packages on top of ones already mentioned:

• texlive-fonts-recommended
• texlive-latex-recommended
• texlive-latex-extra

Otherwise I get errors like this:

Error producing PDF.
! LaTeX Error: File `xcolor.sty' not found.

otherwise you get this:

Traceback (most recent call last):
  File "./wg21/data/filter/wg21.py", line 413, in <module>
    prepare=prepare)
  File "./py/lib/python3.6/site-packages/panflute/io.py", line 236, in run_filters
    prepare(doc)
  File "./wg21/data/filter/wg21.py", line 62, in prepare
    assert(len(codeblocks) == len(texts))
AssertionError
Error running filter wg21/data/filter/wg21.py:
Filter returned error status 1

first run of this on a new system yields:

xelatex not found. Please select a different --pdf-engine or install xelatex

apt installing texlive-xetex on my ubuntu system fixed it

Tables in HTML currently are not of consistent width:

We should try to keep at least a somewhat consistent width for tables.

This table:

+------------+------------------------------------------------------+------------------------------------------------+
| Category   | *reflexpr-operand*                                   | `reflect` Concept                              |
+============+======================================================+================================================+
| Type       | *class-name* designating a union                     | `reflect::Record`                              |
+------------+------------------------------------------------------+------------------------------------------------+
|            | *class-name* designating a closure type              | `reflect::Lambda`                              |
+------------+------------------------------------------------------+------------------------------------------------+

is rendered without cell borders when generating HTML. It would be nice if tables would have cell borders in HTML, just like they have for the C++ standard PDF. This would make complex cells much more readable. (I notice in the reflection TS; CWG will surely appreciate an improvement there!)

§3.1 reads:

Before

switch (x) {
  case 0: std::cout << "got zero";
  case 1: std::cout << "got one";
  default: std::cout << "don't care";
}

After

inspect (x) {
  0: std::cout << "got zero";
  1: std::cout << "got one";
  _: std::cout << "don't care";
}

The 'Before' side is missing break statements after each case's existing statement.

In PDF output, an extra blank line is inserted after a reference if and only if the reference has a URL:

This inconsistent spacing is undesirable. I'm going to assume you like the version with the extra blank line better, so we need to make sure the blank line is always there regardless of whether we have a URL or not. But if you prefer it the other way (no blank line ever) then that's fine too. And if you don't have time to work on this, let me know and I'll send you a PR.

With HTML output I get two single ticks right at the beginning of the doc, see the two ':

&#39;<style type="text/css">'
   'code span.an { color: #1F1C1B; font-style: italic; }'
   'code span.in { color: #006E28; font-style: italic; }'
   'code span.wa { color: #BF0303; font-style: italic; }'
   '</style>&#39;

Unquoting the block fixes it:

diff --git a/template/header.yaml b/template/header.yaml
index e34d2b8..6d22810 100644
--- a/template/header.yaml
+++ b/template/header.yaml
@@ -30,10 +30,10 @@ header-includes:
     ```
   - |
     ```{=html}
-    '<style type="text/css">'
-      'code span.an { color: #1F1C1B; font-style: italic; }'
-      'code span.in { color: #006E28; font-style: italic; }'
-      'code span.wa { color: #BF0303; font-style: italic; }'
-    '</style>'
+    <style type="text/css">
+      code span.an { color: #1F1C1B; font-style: italic; }
+      code span.in { color: #006E28; font-style: italic; }
+      code span.wa { color: #BF0303; font-style: italic; }
+    </style>
     ```
 ---

After updating to the latest master I've started getting the following error:

$ make P0655R0.pdf
pandoc P0655R0.md  ./data/references.md \
       --number-sections \
       --self-contained \
       --table-of-contents \
       --bibliography ./data/index.yaml \
       --csl ./data/cpp.csl \
       --css ./data/template/14882.css \
       --filter pandoc-citeproc \
       --filter ./data/filter/wg21.py \
       --highlight-style ./data/syntax/wg21.theme \
       --metadata datadir:./data \
       --metadata-file ./data/metadata.yaml \
       --syntax-definition ./data/syntax/isocpp.xml \
       --template ./data/template/wg21 \
       --output generated/P0655R0.pdf

fatal error: file read error: "file not found" when accessing "/private/tmp/wg21/data/syntax/language.dtd"
Could not parse syntax definition ./data/syntax/isocpp.xml
make: *** [generated/P0655R0.pdf] Error 67

Any ideas?

I am writing a paper that contains no references to any existing papers, and yet there is an autogenerated "References" section.

Maybe it's juts on my setup, but the HTML style mostly duplicates the PDF one. Meanwhile, P1881 shows that we can do better 🙂

It would be useful to be able to cite something like [@P0443] and have it use the latest revision of that paper. As it is currently it expects the specific revision.

The most recent release of Pandoc 2.10 complains with messages like this:

$ make TEST.html                                                                                                                                                       ❮❮❮
pandoc TEST.md -o generated/TEST.html -d ./data/defaults.yaml --toc-depth 2 \
    --bibliography ./data/index.yaml
Traceback (most recent call last):
  File "/Users/mcypark/Projects/wg21/data/filter/wg21.py", line 511, in <module>
    ], prepare, finalize)
  File "/Users/mcypark/Library/Python/3.7/lib/python/site-packages/panflute/io.py", line 233, in run_filters
    doc = load(input_stream=input_stream)
  File "/Users/mcypark/Library/Python/3.7/lib/python/site-packages/panflute/io.py", line 56, in load
    doc = json.load(input_stream, object_pairs_hook=from_json)
  File "/opt/homebrew/Cellar/python37/3.7.5_3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py", line 296, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/opt/homebrew/Cellar/python37/3.7.5_3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py", line 361, in loads
    return cls(**kw).decode(s)
  File "/opt/homebrew/Cellar/python37/3.7.5_3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/opt/homebrew/Cellar/python37/3.7.5_3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
  File "/Users/mcypark/Library/Python/3.7/lib/python/site-packages/panflute/elements.py", line 1478, in from_json
    raise Exception('unknown tag: ' + tag)
Exception: unknown tag: Caption

Thanks @cjdb for pointing me to it.

I prefer to have a separate git repo for each of my papers. Having git submodule for every repo seems like a waste in this case. Is it possible to provide other means to use this generator?

IIUC 2785e81 broke PDF:

-   'code span.an { color: #1F1C1B; font-style: italic; }'

produces:

\renewcommand{\WarningTok}[1]{\textcolor{rmcolor}{\textit{#1}}}

code span.an \{ color: \#1F1C1B; font-style: italic; \}

which causes:

! LaTeX Error: Missing \begin{document}.

See the LaTeX manual or LaTeX Companion for explanation.
Type  H <return>  for immediate help.
 ...

l.113 c
       ode span.an \{ color: \#1F1C1B; font-style: italic; \}
!  ==> Fatal error occurred, no output PDF file produced!

Help! Kona is coming and I need pdfs! :-)

The tables in LaTeX currently generate something like:

The table is so tight with no separators between rows that it becomes difficult to read. Try to copy the table style used in the standard.

I read most papers on my phone, but your default css has wide margins that make that difficult. Here's a screenshot (using firefox dev tools responsive viewer) of a proposal I'm working on. I extended the length so that more would be visible, but the width is the same as on a Galaxy S9:

Note how there are very few words per line, and how the metadata labels have fallen off the left edge.

I suggest two changes. Either set the margin to 1.5em everywhere, or at least do it within a media query like this (it needs to go after the default body block):

@media screen and (max-width: 30em) {
    body {
	margin: 1.5em;
    }
}

Also, I suggest setting text-align: justify; on the body. It makes a big difference on mobile because the ragged edges are a larger percentage of the total width. But I also think it just looks better for prose even on a desktop.

Here is another screenshot with the modified CSS, or you'd like to quickly try it on any of your devices, this draft paper is using the patched CSS:

If you would prefer, I'd be happy to make a PR with these changes.

I wonder if "first match" semantics should actually be a customisation point?

Beyond the sequential implementaiton you might have an execution policy such as inspect(par) which could execute all matching choices in parallel. You could then implement production systems in the language directly.
par would be an object analogous to an executor rather than a simple enum.

Otherwise I get:

pandoc: generated/Blah.pdf: openBinaryFile: does not exist (No such file or directory)

I've seen several papers that provide a check-box to just hide <del>-ed text from the wording. I think this helps read what the wording is intended to be like, for those papers that change/move around things.

bikeshed has a flag for this.

Is this something mpark/wg21 should provide?

It's expensive and silly.

Add a clickable § or just wrap the whole header in an anchor link tag.

See https://isocpp.org/files/papers/D2529R0.html.

In paper.md: title: generator<T> should have T&& reference_type

In html file: <title>generator`&lt;T&gt;`{=html} should have T&amp;&amp; reference\_type</title>

Title as shown in browser: generator`<T>`{=html} should have T&& reference\_type

The tonytables caption is generated from a blockquote in the span directly. While this is not a problem for pandoc itself, if combined with other filters (pandoc-tablenos) this can lead to problems, if such filters assume a node of type "Plain", such as pandoc-tablenos. It works, if an extra intermediate stage of markdown is used, but not directly.

The following code snippets show my changes for that:

    def quote2plain(elem):
        result = pf.Plain()
        result.content = elem.content[0].content # switch from BlockQuote Para content to Plain
        return result 
    for elem in table.content:
        if isinstance(elem, pf.Header):
            if not isinstance(header, pf.Null):
                warn(header)

            if first_row:
                header = pf.Plain(*elem.content)
                if 'width' in elem.attributes:
                    width = float(elem.attributes['width'])
            else:
                warn(elem)
        elif isinstance(elem, pf.BlockQuote):
            caption = quote2plain(elem)
            kwargs['caption'].content.append(caption)
        #...

Feel free to close this issue immediately.

I have a paper whose metadata has:

title: "Inheriting from `std::variant`"

This gets rendered in HTML as:

<title>Inheriting from </title>

Although this does the right thing for the actual heading;

<h1 class="title" style="text-align:center">Inheriting from <code class="sourceCode cpp">std<span class="op">::</span>variant</code></h1>

I can fix this by adding a separate pagetitle element with "Inheriting from std::variant" (no markdown), but would be cool if that happened automatically.