leanprover / doc-gen4 Goto Github PK

The documentation should show the constructor name in structures, at least when you manually specify it using mk' ::. For example https://leanprover-community.github.io/mathlib4_docs/Init/Prelude.html#ULift should include up ::.

Zulip discussion: https://leanprover.zulipchat.com/#narrow/stream/217875-Is-there.20code.20for.20X.3F/topic/linear.20maps.20of.20polynomials/near/267925047

Mathlib makes massive use of doc strings and mathematical symbols in its doc strings, we should be able to render this properly in the browser just like the current doc-gen can.

We should link back from the rendered LeanInk to the types. This requires changes on both doc-gen and LeanInk. I also noticed that LeanInk does not mark things like + as something hoverable so operations won't be clickable with the current implementation.

In doc-gen core we can already render the comments properly as markdown, can we achieve this in ink as well?

Certain folders in the run of the current doc-gen on mathlib4 have associated HTML pages, for example, mathlib itself:
https://leanprover-community.github.io/mathlib4_docs/Mathlib.html
and Mathlib/Tactic/Linarith:
https://leanprover-community.github.io/mathlib4_docs/Mathlib/Tactic/Linarith.html
Not all folders have these HTML pages; I don't understand the selection mechanism.

In principle I suppose we could add text to these pages and make them useful. However, I think it is better not to have them at all, because in the left sidebar display those folders are displayed with HTML links to the associated pages, which makes it hard to expand those folders -- one has to click on the small arrow to the left of the folder rather than the folder itself.

Could these HTML pages simply be disabled?

Currently all entries in a module are unsorted, they should be sorted by the line number inside of their module.

High-severity issue :)

If you enter something into the search box, the autocompletions will only show up to a limited number of results. If you hit enter it goes to google site search which is an entirely different set of results. Would it be possible to just see the full set of autocomplete results for a given query?

https://leanprover.zulipchat.com/#narrow/stream/270676-lean4/topic/.E2.9C.94.20docs4.23/near/271780657

https://leanprover.zulipchat.com/#narrow/stream/113488-general/topic/Making.20mathlib.20more.20readable.20.28was.3A.20Improving.20the.20mathlib.20.2E.2E.2E/near/291460564

https://leanprover.zulipchat.com/#narrow/stream/113488-general/topic/Making.20mathlib.20more.20readable.20.28was.3A.20Improving.20the.20mathlib.20.2E.2E.2E/near/291485777

Implementation idea:

1. The individual JSON of each module contains:

• A list of used theorems per theorem
• A map from theorem names to a counter for how often this name was used

2. In the index step this information is accumulated into the final JSON in such a way that the following things can be calculated in JS with little runtime:

• The list of theorems a certain theorem used, sorted by use counter, the theorem with the least use counter first. This is so theorems that are special for this declaration show up first
• The list of theorems that use a certain theorem, sorted by use counter, here the theorem with the highest counter first. This is so the significant results can be visited quickly.

Right now given a type, typeclass and the automatic (or user provided) names it is possible to check whether the type has an instance for this typeclass by:

1. Searching for the typeclass
2. Opening all of its instances
3. Looking through the instances and hopefully hitting the one one is looking for by C-f-ing the name of the type

It would be nicer if instead we could discover all instances that related to a type right at the types declaration itself.

The navigation bar makes the generation of documentation in a step wise manner annoying. We always need to know all modules that might ever be there even for just generating the index.html. docs.rs uses an approach that only shows information about the currently clicked module, this might make things easier for generation.

In doc-gen3, if I follow a link to some declaration (e.g.
https://leanprover-community.github.io/mathlib_docs/algebra/group/units.html#units
) then the path to that declaration (algebra/group/units) is unfolded in the left navbar. Likewise if I search for that declaration (in that case, it keeps any previously-unfolded things open but also unfolds the thing being navigated to).

In doc-gen4, if I follow a link to some declaration (e.g.
https://leanprover-community.github.io/mathlib4_docs/Mathlib/Algebra/Group/Units.html#Units
) then the path to that declaration (Mathlib/Algebra/Group/Units) is not unfolded; same if I search for the declaration.

I find the doc-gen3 behaviour more convenient; could it be copied over?

The include_str command is relative to where you run the build command, this lead to such error when users try to import DocGen4:

info: ./lean_packages/doc-gen4/././DocGen4/Output/Base.lean:50:24: error: "./static/nav.js" does not exist as a file
./lean_packages/doc-gen4/././DocGen4/Output/Base.lean:51:27: error: "./static/search.js" does not exist as a file
./lean_packages/doc-gen4/././DocGen4/Output/Base.lean:52:34: error: "./static/mathjax-config.js" does not exist as a file

This will reduce dependencies on pure C code and also allow us more customization in the way we want to render markdown from comments.

I often press space in my web browser to go a page down, and shift+space to go a page up.
These shortcuts don't appear to be working on pages such as https://leanprover-community.github.io/mathlib4_docs/Mathlib/Order/Basic.html#PartialOrder.lift when I previously clicked a link in either nav column to the left or to the right.

It's a small inconvenience, but I'd prefer it if the navbars wouldn't steal the "input focus" so that my browser always does the PageDn in the middle column.

Syntax declarations are shown with their auto generated name as well as their parser as value right now. We could instead try pretty print the parser declaration and denote them explicitly as syntax with the proper syntax instead of a weird def that's just dangling around.

Every doc-gen page has a list of definitions (see right half: https://leanprover-community.github.io/mathlib_docs/init/core.html#prod).

Users may want to have customized content in their index page.

A simple solution is to add a Cli FLAG that allows users to specify a .html or .md file.

The more comprehensive method is to have a configuration file similar to lakefile.lean, so that people can customize everything from documentation title to additional navbar pages (like that of mathlib).

Showing equations of definitions like here https://leanprover-community.github.io/mathlib_docs/init/core.html#nat.add

Declaration modifications that are not shown in the output yet:

• noncomputable
• unsafe
• partial
• abbrev
• class inductive
• attributes

Compare:

• Lean 4: exists_eq
• Lean 3: exists_eq

Only the Lean3 versions let you inspect the binder symbol in the docs. While for Existss this is obvious, for other binders the hover text is very useful.

It's possible this requires a patch to Lean4 itself; the Lean 3 patch was leanprover-community/lean#781.

For me (Firefox on Mac) the declaration abs_eq_zero displays without line wrapping in the list of instances. I don't seem to have Github permissions to add a screenshot, unfortunately.

In the corresponding declaration in doc-gen3, there is line wrapping.

Integrating with https://github.com/insightmind/LeanInk would be very useful for proof exploration.

Every doc-gen declaration has a source link attached (see e.g.: https://leanprover-community.github.io/mathlib_docs/init/core.html#nat), since doc-gen4 should support arbitrary projects we basically want the same feature but with a custom base URL.

Currently, doc-gen4 segfaults when run an older Lean toolchain. Is it possible to detect this case and show a user-facing error?

Running doc-gen in some sort of "Book Mode" where it generates a web-page that:

1. Primarily focuses on the LeanInk instead of the doc part
2. Makes the LeanInk searchable
3. Somehow ™️ has book like features such as an index etc., simliar to mdbook
4. But can still link back to the doc
   Would be nice for documentation like the meta programming book or functional programming in lean.

Fields of structures/classes and constructors of inductive types are not linked to correctly. Examples:

https://leanprover-community.github.io/mathlib4_docs/find/?pattern=Nat.zero#doc
https://leanprover-community.github.io/mathlib4_docs/find/?pattern=Group.toDivInvMonoid#doc

Right now the tool already generates links to inductive constructor names and structure fields, these links should actually point to the correct positions.

Show structure fields properly like here: https://leanprover-community.github.io/mathlib_docs/init/core.html#prod.

The mathlib docs support the prefers-color-scheme media query, and supports dark or light mode depending on the response. However, the mathlib4 docs do not change their color. For example, for prefers-color-scheme: dark
docs3:

docs4:

unif_hint like in https://github.com/leanprover/lean4/blob/189f4bd372d86d09f9299ad58c13fe661a327444/src/Init/Hints.lean should be rendered here https://leanprover-community.github.io/mathlib4_docs/Init/Hints.html

Render docstrings on inductive types constructors, don't forget about class inductive.

The linkifier doesn't work well when linking to names with special characters, including primes

Examples:
https://leanprover-community.github.io/mathlib4_docs/find/?pattern=MonoidHom.compl₂#doc
https://leanprover-community.github.io/mathlib4_docs/find/?pattern=pow_succ'#doc

See discussion on Zulip here (point 2) and here. This makes it easier to browse deeply nested file trees (as is common in mathlib); it was the docs3 behaviour.

With the current navbar tree generation lots of links are actually broken/don't exist. The navbar names that do not actually point to real files should not be marked with a link to those files.

In the doc-gen for mathlib3, the left sidebar looks like this:

core
▸ data
▸ init
▸ system

mathlib
▸ algebra
▸ algebraic_geometry
▸ algebraic_topology
▸ analysis
▸ category theory
...

That is, the repo we have actually run doc-gen on (mathlib) and its dependencies (core) are displayed as headings and all of their top-level folders are displayed afterwards in a list. I like this feature because I spend a lot of time browsing mathlib and this saves me a click. Mathlib hierarchies also get pretty deep, so having all the mathlib top-level folders left-justified also saves a few indentation characters which comes in handy deeper down.

In the doc-gen for mathlib4, the left sidebar looks like this:

▸ Init
▸ Lean
▸ Mathlib
▸ Std

I can see why @hargoniX made this choice, because now the dependencies (Init, Lean and Std) all have many top-level folders, and displaying all of these would push the top-level folders from the repo of interest (Mathlib) far off-screen!

It's still annoying to have the extra click, though (and this is exacerbated by the issue in #99 causing the valid clickable region to be a single unicode symbol ▸).

Here is a more complicated proposal which I think would be the best of both worlds: first display the dependencies, with just their names, then display all the top-level folders of the repository of interest. So

▸ Init
▸ Lean
▸ Std

---

Mathlib
▸ Algebra
▸ CategoryTheory
▸ Combinatorics
▸ Control
...

Alectryon can render pretty versions of data structures like rbtrees,
integrating this feature from the compiler: leanprover/lean4#1238
might allow us to do the same (https://alectryon-paper.github.io/snippets/rbt.html)

This requires work on:

• The LSP's semantic highlighting
• LeanInk to expose it to doc-gen
• doc-gen to render it

It would be good to forward-port the features from:

• leanprover-community/doc-gen#172
  Instance lists for builtin types, like we already have for inductive types
• leanprover-community/doc-gen#173
  Distinguishing between different ways of printing pi types. This probably should adapt to the Lean4 changes around pretty-printing.
• leanprover-community/doc-gen#174
  Linkifying these special types

The PRs above are all centered around this page: https://leanprover-community.github.io/mathlib_docs/foundational_types.html

In the documentation for MulEquiv.withOneCongr, the declaration Equiv.optionCongr is referenced, but there is no automatic hyperlink to that declaration in the docs.

The corresponding documentation in doc-gen3 does have the hyperlink.

The same bug occurs elsewhere, e.g. a missing hyperlink to CanonicallyOrderedCommSemiring in this module docstring, and OrderIso.mulLeft where it misses the link to OrderEmbedding.mulLeft and instead links to a locally-defined mulLeft.

In doc-gen3, the path of the file currently displayed appears at the top of the right sidebar, with linebreaks preferentially inserted at the . between a folder and its subfolder. In doc-gen4 the linebreaks simply fall where they may, which is a small loss in readability. Could the doc-gen3 behaviour be implemented here?

Compare, e.g.,
https://leanprover-community.github.io/mathlib_docs/algebra/group_power/basic.html
https://leanprover-community.github.io/mathlib4_docs/Mathlib/Algebra/GroupPower/Basic.html

If I understand correctly, certain fields that one might want to set in the documentation on a repo-by-repo basis are currently hard-coded in doc-gen4. For example,

• the title displayed in the top left corner (hard-coded as "Documentation", mathlib3 docs say "mathlib documentation" with a link on "mathlib" to the leanprover-community website)
• the central panel of the index page (hard-coded as "What is up?" 😃), mathlib3 docs have several paragraphs introducing mathlib and giving links to the mathlib, Lean-fork and doc-gen versions that the build is based on
• the HTML <title> tag (hard-coded as eg "Mathlib.Algebra.Group.Basic", mathlib3 docs say "algebra.group.basic -- mathlib docs")

What would be the best way to allow this to be set on a repo-by-repo basis? Maybe a fixed format for a configuration file which could live in the top level of the associated repository?

(Note: maybe with that method it wouldn't be possible to have these fields to contain links to the mathlib/Lean/doc-gen commits used to generate the docs, the way that the central index panel does in the mathlib3 docs. So maybe we should also find another place to display that information.)

In the mathlib3 docs, a directory which has been unfolded by clicking on it remains unfolded after you click on any other link. This is helpful, for example, if you wish to browse one by one through the files in some deeply-nested directory.

In docs4 it seems that after clicking any link, all directories get folded back again. Can this be changed?

In this page, the left sidebar is missing and definitions after Unicode.bidiMirroringStr is quite small.

Looking at the source code, the nav bar element is missing and there are nested and tag surrounding the definitions. This bug exists in earlier version of doc-gen4 so it's not the problem of docstring. But I also fail to find that in doc-gen4 source code.

Show the constructors of inductive types like here: https://leanprover-community.github.io/mathlib_docs/init/core.html#nat.

A type based search like Hoogle.

In doc-gen type classes are represented as structures with the class attribute and instances as defs with the instance attribute, we should decide whether we want to do this the same way or distinguish them visually (and textually) as is done right now. Note that the current approach in this repo isn't integrated with the CSS yet so it doesn't work perfectly, this would have to be done as well if we decide against the current doc-gen approach.

Furthermore type class instances aren't listed at the type class definition yet and type class fields should be rendered like structure ones.

Every doc-gen page has a list of modules that import the current one/are imported by the current one (see top right corner: https://leanprover-community.github.io/mathlib_docs/init/core.html#prod).

The page at https://leanprover-community.github.io/mathlib4_docs/find/ohno

gives me