Make iset.mm typesetting match set.mm about set.mm

[david-a-wheeler]

I like the idea of putting the typesetting instructions in a single file for inclusion by both. I can't think of a case where it is important to have different type setting.

[nmegill]

Having a common typesetting file is problematic because 'verify markup' requires that each htmldef have a $c/$v and vice-versa. set.mm has symbols that iset.mm doesn't and vice-versa. The software currently allows only one $t comment. Because file inclusion statements $[...$] are ignored inside of comments, we can't separate out the htmldefs common to both .mms using file inclusions.

Even if the common htmldefs mismatch, it is usually just a minor cosmetic issue. In this case we're talking about 2 symbols that are trivial to edit manually, and such changes are rare. We can think about allowing multiple $t's in metamath.exe but that won't happen right away.

An immediate solution is to arrange the htmldefs so that the ones common to both set.mm and iset.mm are at the beginning of the $t statement, followed by the ones that are custom to each .mm. Then synchronizing them would be a simple copy/paste guided by comments around the initial htmldef group. This rearrangement would be needed anyway if someday we allow multiple $t's, so it wouldn't be wasted effort.

Norm

[jkingdon]

I know it is more than two symbols (one very visible one, at least in some fonts, is "ph" which is 𝜑 in set.mm and φ in iset.mm) . Part of why I made an issue and am asking for volunteers is that it would take a bit of time to go through the files and figure out what to copy where; I'm not sure how many symbols are at issue.

For all the reasons Norm gives, though, copy-paste is probably the best way to do this. I'm not exactly opposed to something fancier, just that it isn't super high on the list of things I find inconvenient. And it is true that the typesetting doesn't change super often.

[david-a-wheeler]

You can get a list of typeset values with "python scripts/typeset.py --list" - that may make this easier.

[jkingdon]

From @nmegill :

Around 2016 we started using XITS fonts, which solved numerous browser Unicode incompatibility problems. Before XITS, we worked around some problems by using sans-serif that you'll see with some of the iset.mm symbols (e.g. the althtmldef for "|-") in the $t comment. In addition, XITS implements "math italic" commonly used in math books for Roman and Greek letters; it is subtly different from regular italic.

We updated the $t in set.mm to use XITS. In iset.mm, we updated the $t "boilerplate" but not the althtmldefs.

To see an example of the differences, compare
http://us2.metamath.org:88/mpeuni/abid.html
and
http://us2.metamath.org:88/ileuni/abid.html
You'll see that the display for the math symbols "|-", "x", "e.", "ph" are slightly different. As a quick local experiment, I changed the althtmldefs for these 4 symbols in iset.mm to match those in set.mm, and the math symbols on the two pages became identical. This shows that the XITS font is being correctly recognized, so all that has to be done is to update the althtmldefs to match set.mm. (It is probably more practical just to replace the entire htmldef/althtmldef/latexdef triples with the ones in set.mm.)

[benjub]

On a related note, maybe it is time to switch to STIX2 ? As I noted in https://groups.google.com/d/msg/metamath/X7SO7EBGUjw/B0gop2muCwAJ XITS was created to fix shortcomings of STIX, but now that STIX 2 is out, it might be preferable. See for instance http://us2.metamath.org/mpeuni/ccchat.html

[nmegill]

There is also a problem with the htmldefs in iset.mm that use images. The w3.org validator has started to require that every image be accompanied by ALT text. Because of this, essentially every ilegif page (e.g. http://us2.metamath.org:88/ilegif/abid.html) has dozens of errors when you click on "W3C validator" in the lower right corner, obscuring any "real" errors that might be present. I fixed this in set.mm, but iset.mm, nf.mm, and ql.mm need to be updated.

For example, below are the htmldefs for the symbol "ph" in iset.mm and set.mm:

iset.mm (bad):

htmldef "ph" as
    "<IMG SRC='_varphi.gif' WIDTH=11 HEIGHT=19 TITLE='ph' ALIGN=TOP>";

set.mm (good):

htmldef "ph" as
    "<IMG SRC='_varphi.gif' WIDTH=11 HEIGHT=19 ALT=' ph' TITLE='ph'>";

Again, this can be corrected by copy/pasting the corresponding htmldef/althtmldef/latexdef triple from set.mm, so the font issue with the althtmldefs I mentioned above will be corrected at the same time.

Some notes:

• When both ALT and TITLE are present, in most browsers the TITLE gets used as the tool tip displaying the ASCII symbol, and the ALT gets used when you copy/paste into a text editor. For the latter to be useful, we add a space in front of each ALT so that the symbols don't collide when pasted, allowing people to copy/paste directly into iset.mm. I think this feature is still used by some people and is one of the reasons we've kept the GIF version of web pages. The feature isn't perfect because some htmldefs don't have images, but it is close enough to be practical.
• The ALIGN=TOP field is no longer required (and may be detrimental IIRC, shifting the GIF up by one pixel compared to normal text, in some browsers). Alignment is now handled with the CSS directive "img { margin-bottom: -4px }" at the top of each page.

[nmegill]

On a related note, maybe it is time to switch to STIX2 ? As I noted in https://groups.google.com/d/msg/metamath/X7SO7EBGUjw/B0gop2muCwAJ XITS was created to fix shortcomings of STIX, but now that STIX 2 is out, it might be preferable. See for instance http://us2.metamath.org/mpeuni/ccchat.html

@benjub Have you verified that STIX 2 produces a correct ccchat (I assume CC with a hat)?

@david-a-wheeler Since you were the one who researched and brought XITS into Metamath (thank you!), what do you think of this proposal?

[benjub]

@nmegill It looks on my computer like STIX2 correctly produces \hat{\mathbb{C}} A safe thing to do could be to temporarily put a page

http://us2.metamath.org/mpeuni/mmascii-STIX2.html

which is just a copy of mmascii.html but using STIX2. This would allow people to compare both, on various browsers and OS's. I think one simply has to replace the lines

  @font-face {
    font-family: XITSMath-Regular;
    src: url(xits-math.woff);
  }
  .math { font-family: XITSMath-Regular }

with the corresponding STIX2 versions. I'm not sure whether it is

  font-family: STIX2Math;
  src: url(STIX2Math.woff2);

or a variant (e.g. STIXTwoMath)...

[benjub]

Concerning the GIF thing: I am not sure if this is worth the work, regarding how the GIF version may be deprecated in the future?
I would be happy to have two versions for each explorer: /mpeuni for unicode, and /mpeascii for plain ascii. The generation by the MM program of the latter would be easy. In the database files, only the althtmldef (which could be renamed unicodedef or something like that for clarity) would be needed. I noticed that the new "htmldef's" (the GIF definitions) are more and more written cavalierly anyway. What do people think? (or maybe this is a topic for the google discussion group?)

[nmegill]

@nmegill It looks on my computer like STIX2 correctly produces \hat{\mathbb{C}} A safe thing to do could be to temporarily put a page

http://us2.metamath.org/mpeuni/mmascii-STIX2.html

which is just a copy of mmascii.html but using STIX2. This would allow people to compare both, on various browsers and OS's. I think one simply has to replace the lines

..............

  font-family: STIX2Math;
  src: url(STIX2Math.woff2);

I found a couple STIX2Math.woff2 files on GitHub and elsewhere but they have different sizes. Do you know the link for the latest official version?

or a variant (e.g. STIXTwoMath)...

Why would we us a variant?

[benjub]

Why would we us a variant?

Sorry, I meant: use the main STIX2 package, but I simply do not know how it is named exactly. On the website https://www.stixfonts.org/ there is a section "How to install" which links to https://github.com/stipub/stixfonts I do not know which to choose among otf, woff, woff2.

[nmegill]

Concerning the GIF thing: I am not sure if this is worth the work, regarding how the GIF version may be deprecated in the future?
I would be happy to have two versions for each explorer: /mpeuni for unicode, and /mpeascii for plain ascii. The generation by the MM program of the latter would be easy. In the database files, only the althtmldef (which could be renamed unicodedef or something like that for clarity) would be needed.

I think this was brought up before in the Google Group but I can't find it. Anyway, the "*def" variants are is somewhat tightly hardcoded into the program (affecting lots of things like 'verify markup') in a way that's not easy to extend. I.e. adding a new *def type can't be done as a quick experiment. If I did that I would want to be convinced it is permanent and is really what people highly demand.

If someone wants to do a quick experiment to see what it would look like, a script could just replace all htmldef assignments with their ASCII tokens. That would be much simpler than adding a new *def type to the program.

BTW adding a new directory like mpeascii would add another 30K files and isn't something I'd want to do lightly.

I noticed that the new "htmldef's" (the GIF definitions) are more and more written cavalierly anyway. What do people think? (or maybe this is a topic for the google discussion group?)

Yes, our trend has been not to use GIFs when the token and symbol is alphanumeric, especially if the symbol and its ASCII token are identical. That is fine, because copy/paste will still work. When graphic symbols are used, we typically use a new GIF symbol from the "GIF and PNG Images for Math Symbols" page or combine a couple of them. Occasionally we have to create a new GIF, but it has become rare.

Using GIFs for graphic symbols hasn't been followed perfectly, but the missing GIF cases tend to be in more specialized stuff that are copy/pasted this way less frequently. Ideally someone would go through them at some point to fix missing GIFs, but it hasn't been a problem and I think it's relatively low priority.

Further discussion should probably go into the Group, ideally added to the thread about this if it can be located. What we say here will be lost into obscurity...

[benjub]

@nmegill You're right, I'll write a new topic on the google group summarizing my proposal (I do not remember of a post on the google group about this, but you and I had already discussed this topic privately). I'll wait a few days to give a chance to have mmascii-STIX2.html up by then.

[david-a-wheeler]

I don't object to STIX2 per se, but we should test it first before switching.

[benjub]

@nmegill or @david-a-wheeler, is it possible to put on the us2/mpeuni website the page mmascii-STIX2.html which is simply mmascii.html with the following modifications:
Change

@font-face {
  font-family: XITSMath-Regular;
  src: url(xits-math.woff);
}
.math { font-family: XITSMath-Regular }

to (something like):

@font-face {
  font-family: STIX2Math;
  src: url(STIX2Math.woff2);
}
.math { font-family: STIX2Math }

On the website https://www.stixfonts.org/ there is a section "How to install" which links to https://github.com/stipub/stixfonts. I do not know which to choose among otf, woff, woff2.

After some testing, we can switch globally to STIX2, and after that @jkingdon can replicate this to iset.mm

[jkingdon]

After some testing, we can switch globally to STIX2, and after that @jkingdon can replicate this to iset.mm

Sure if I get to it or even better someone else might find the time before I do.

[nmegill]

@nmegill or @david-a-wheeler, is it possible to put on the us2/mpeuni website the page mmascii-STIX2.html which is simply mmascii.html with the following modifications:
Change

@font-face {
  font-family: XITSMath-Regular;
  src: url(xits-math.woff);
}
.math { font-family: XITSMath-Regular }

to (something like):

@font-face {
  font-family: STIX2Math;
  src: url(STIX2Math.woff2);
}
.math { font-family: STIX2Math }

I did this.
http://us2.metamath.org/mpeuni/mmascii-STIX2.html
(vs. http://us2.metamath.org/mpeuni/mmascii.html)
I noticed that the font is slightly different from XITS e.g. the "z" is closer to the table border. To check if that causes problems, I also created
http://us2.metamath.org/mpeuni/mmdefinitions-STIX2.html
(vs. http://us2.metamath.org/mpeuni/mmdefinitions.html)
so it can be checked visually for possible spacing problems.

To use the font everywhere, we would modify the "htmlcss" definition in the $t statement of set.mm.

On the website https://www.stixfonts.org/ there is a section "How to install" which links to https://github.com/stipub/stixfonts. I do not know which to choose among otf, woff, woff2.

It would be nice to figure this out. Perhaps someone can make local versions of mmascii-STIX2.html and mmdefinitions-STIX2.html with the different font files, and look at them on different browsers.

BTW for future reference, mmascii.html and mmdefinitions.html are created with 'show statement ! /alt_html' (see end of 'help write theorem_list). (In '!', the '' triggers these files to be generated, and '!' prevents a match to any specific statement so that we don't generate the individual theorem pages; '@' would also work.)

After some testing, we can switch globally to STIX2, and after that @jkingdon can replicate this to iset.mm

This just requires changing "htmlcss" as in set.mm (copy/paste from set.mm should work I think).

It also requires that I duplicate the font file in the ileuni directory. (At least in 2016, some browsers wouldn't recognize font files in a different directory, which was apparently related to some kind of licensing issue: https://groups.google.com/d/msg/metamath/uvW5-xV8iEw/uOPsVHxSAAAJ)

[nmegill]

BTW for future reference, mmascii.html and mmdefinitions.html are created with 'show statement ! /alt_html' (see end of 'help write theorem_list). (In '!', the '' triggers these files to be generated, and '!' prevents a match to any specific statement so that we don't generate the individual theorem pages; '@' would also work.)

means:

BTW for future reference, mmascii.html and mmdefinitions.html are
created with 'show statement *! /alt_html' (see end of 'help write
theorem_list).  (In '*!', the '*' triggers these files to be generated,
and '!' prevents a match to any specific statement so that we don't
generate the individual theorem pages; '*@' would also work.)

[benjub]

Thanks @nmegill. I can see some slight differences but I'm not sure which one I prefer. My main hope was to fix the characters CChat and RRhat, but they are as bad as before, while CCbar and RRbar are worse than before :( When I save the page as html and then open it, locally, with my browser, they appear much better, but I'm not sure why (the browser might be using a "system font", but I'm not sure). Waiting for other opinions.

[nmegill]

When I save the page as html and then open it, locally, with my browser, they appear much better, but I'm not sure why (the browser might be using a "system font", but I'm not sure).

They look the same locally and remotely for me, in Firefox and Chrome on Windows 7. Are you sure the .woff2 file is in the same directory as the html? Also, sometimes shift-reload is needed if there is a caching issue.

[benjub]

You're right: I did put the STIX2Math.woff2 file in the same directory as the downloaded mmascii-STIX2.html file and it rendered as on the website. When the woff2 file is not there, then the browser probably "falls back" to some "default" or "system font", which looks a bit more rustic, but which ironically renders CChat, RRhat, CCbar, RRbar correctly. I'll try to figure out what font it is.

[benjub]

To answer my above comment: the browser falls back to the "browser's default font", which in my case (Firefox on Debian) is DejaVu Sans Serif. Therefore, it might be STIX and XITS which have a problem with combined characters (https://en.wikipedia.org/wiki/Combining_Diacritical_Marks).

I don't have a strong opinion on this, and I know little about fonts, but it looks like if STIX2 is ok, it's probably better to switch to it (see https://tex.stackexchange.com/questions/227216/stix-versus-xits). But it looks like the added value of STIX/XITS compared to more standard fonts is their use with LaTeX (I haven't looked into that), and now many fonts encode all/most of the unicode characters defined in set.mm anyway (for instance, DejaVu, the default of my browser). A computer savvy person (@david-a-wheeler ?) will know better.

[david-a-wheeler]

I did a quick comparison of the STIX2 test results ( http://us2.metamath.org/mpeuni/mmascii-STIX2.html ) and the current XITS results (currently http://us2.metamath.org/mpeuni/mmascii.html ).

Personally, I like how theta is rendered better currently; STIX2 seems worse. More specifically, I think the STIX2 theta is thin and less good-looking. But this is the problem with comparing fonts, for the most part it's very subjective. Both fonts are reasonable choices.

CChat looks terrible on both using Firefox on Windows, so that isn't a distinguishing characteristic between tehn.

We don't have to use either STIX2 or XITS, of course. I chose XITS originally because XITS was specifically designed to cover all math symbols (so we were unlikely to ever add a symbol not already in XITS), the original STIX had problems, and STIX2 as not out yet. DejaVu Sans Serif is a plausible alternative and is freely available. However, I think we'd better do a lot more testing before choosing a font not specifically designed for math, to ensure that it meets our needs, and there's always the worry that someone will want to use an obscure math symbol (which I think is is more likely to be covered by math-specific fonts). That said, far more people work on general-purpose fonts, so maybe general-purpose fonts would actually meet our needs better.

BTW, I recommend specifically including Firefox in your tests. It's pickier about font locations than some other browsers, I believe for security reasons, so you increase the likelihood of detecting problems by using Firefox.

[jkingdon]

I realize there are a lot of comments on this issue. If there are any problems not fixed by #2183 , that anyone would like to track in a github issue, I'd suggest making new issue(s) for the outstanding item(s).