martinthomson / i-d-template

A template for IETF internet draft git repositories

Internet Draft Template Repository

The contents of this repository can be used to manage the editing of an Internet Draft.

This tool provides many features.

Getting Started

You need a GitHub account.

The template repository is the easiest way to get started. You don't need any additional software.

If you want to use command-line tools, make sure you have the necessary software installed. Manual setup is mostly automated.

Work.

Submit the final product.

i-d-template's Issues

make submit not working

I think after you fixed #10, you need to try make submit. I get:

make: *** No rule to make target `draft-ietf-unicorn-protocol-00.txt', needed by `submit'.  Stop.

I also tried tagging.

Yes, xml2rfc.{tools.}.ietf.org is flaky sometimes, but unicorn-wg is flakier...

Can we move over to https://xml2rfc.tools.ietf.org/public/rfc now?
(Note the "s".)

.gitignore for /lib needed on gh-pages

make gh-pages was failing for me due to untracked files under lib/. I added a .gitignore file with an entry to lib/ on the gh-pages branch in order to suppress that error. This fixed the make.

(Or maybe this isn't needed and I did something wrong? First time I'm using this toolchain.)

Can't seem to build with list

Hey! I am getting errors on the build due to using . See below:

ERROR: Unable to validate the XML document: draft-rooney-marnew-report-latest.xml draft-rooney-marnew-report-latest.xml: Line 38: Element section content does not follow the DTD, expecting ((t | figure | texttable | iref)* , section*), got (t section section t list t)

I guess I ambeing a n00b again, but any help would be great!

If it hekps I followed your syntax from https://github.com/webpush-wg/webpush-protocol/blob/master/draft-ietf-webpush-protocol.xml

Missing copyright notice

There should be a copyright notice as part of this project so that people don't get in trouble.

Explain use of git tags to generate file names

Under the heading of "Submitting Drafts", the README.md says about generating files to submit:

Make a submission version of your draft. The makefile uses git tags to work out what version to create.

When I installed this on my system for the first time, I wasn't sure what to do here, i.e. should I tag the repo before running make submit? I didn't... and found that it correctly generated a -00 file.

But it's not clear to me how I get it to generate a -01 file. I'm guessing from the text below this line that I need to tag it with the draftname-latest and THEN run make submit.

Perhaps a start would be to add something like:

Make a submission version of your draft. The makefile uses git tags to work out what version to create. The first time you run the command without the repository being tagged it will generate a -00 draft.

And then perhaps some text about when you should tag your repo in what form in order to generate next files.

(I'd submit a PR on this with some text... but I don't understand enough about how it works yet to do so!)

gh-pages clone error in Travis

The error I get is: "attempt to fetch/clone from a shallow repository" after the git clone -q -b gh-pages . /tmp/ghpages****

By doing a very short google search, it seems to be a Travis problem linked to the number of commits existing, but I was wondering if there was a way to patch this in the ghpages script itself.

This is a build failing, while this one passed.

(make ghpages manually does work)

Include instructions on what to install

Yo Martin! I have an error when building gh-pages with md files (not xml):

Deoxys:RedMage_SystemArchitecture Ifrit$ make ghpages
kramdown-rfc2629 draft-ietf-redmage.md > draft-ietf-redmage.xml
/bin/sh: kramdown-rfc2629: command not found
make: *** [draft-ietf-redmage.xml] Error 127
rm draft-ietf-redmage.xml

XML title on multiple lines fails

If there is a NL in the middle of the title, then the script fails.

I will look at this based on comments from Martin to look at a fix

Multi-draft repos

I would like to have multiple draft-*.md files in my repo.

"push" CI job fails while "pr" CI jobs succeeds

core-wg/coap-tcp-tls#126

I don't understand why the push CI job uses a directory it doesn't seem to create (/tmp/ghpages1834/abnf-fix/, where abnf-fix is the name of the branch), while the pr CI job just sails through.

Not getting ghpages updated on Travis

The problem is that Travis_Pull_Request is set to false. You can see the environment dump at
https://travis-ci.org/sacmwg/vulnerability-scenario/builds/121561135#L617

example.md has drifted...

Yo, Martin. example.md and example.xml seem to be very very different (the .md about STUN and the xml very much looking like a template). Can you put the BS template back in example.md and then perhaps peel off the STUN one as a rich example?

Push built versions of branches of the main repo to directories on gh-pages

i.e., push to in_progress and you get a preview at https://<user>.github.io/<repo>/in_progress/

Getting a live view of what is going on for a branch is useful.

Add scroll-boundary-behavior to CSS

No one implements this just yet, but it's probably a good idea for the TOC on wide screens.

https://github.com/WICG/scroll-boundary-behavior

TRAVIS_PULL_REQUEST is "false", not the empty string if on master ➔ CI_IS_PR is always true

https://docs.travis-ci.com/user/environment-variables/
"Environment Variables - Travis CI"

says:

TRAVIS_PULL_REQUEST the pull request number if the current job is a pull request, or “false” if it’s not a pull request.

And that is also what I see.

Unfortunately, id.mk checks for an empty string:

ifneq (,$(TRAVIS_PULL_REQUEST)$(CI_PULL_REQUESTS))
  CI_IS_PR = true
else
  CI_IS_PR = false
endif

So CI_IS_PR is always true on Travis.

gh-pages pollution

Check out the gh-pages branch of the QUIC repo. The old versions of the documents pre-rename are there, as well as leftovers from every defunct branch that ever built gh-pages. Should there be some pruning to make sure we drop old build artifacts, or does it not matter since those things are going to live forever in git anyway, whether they're in the current state or not?

Travis build: fatal: attempt to fetch/clone from a shallow repository

The most recent Travis build for TCP-TLS complains that cloning the gh-pages fetch is not possible, as it is shallow.
Is this something we broke in some way?

Save issues as a build artifact

This is probably abusing the artifacts capabilities of the CI systems, but we could snapshot issues with each build.

Missing support for drafts authored with outline.el (emacs org mode)

Currently supports Markdown and XML, but not Outline - WTF? ;)

Travis build failing in ghpages

Hi Martin,

Since a couple of days ago the Travis build is failing in the make ghpages:

Error: No gh-pages branch, run make -f lib/setup.mk setup-ghpages to initialize it.
make: *** [fetch-ghpages] Error 1

Do you have any idea why this could be happening? Any help would be appreciated!

support mmark?

Could you add docs/support for using mmark[*] as well? See https://github.com/miekg/mmark and https://miek.nl/tags/mmark/

Macs have an old, terrible version of bash

Two problems:

Negative subscripts "${x[-1]}" don't work.

printf doesn't seem to work either, but that might be downstream from the first problem.

make ghpages fail

Hi Martin,

I'm having a problem I don't understand with the Travis build, that appeared yesterday:
https://travis-ci.org/core-wg/oscoap/builds/213763017

Also when I try to run make ghpages manually I get this errors:

git fetch -q origin gh-pages:gh-pages
git clone -q -b gh-pages . /tmp/ghpages1548
/usr/bin/sh: command substitution: line 1: syntax error near unexpected token `('
/usr/bin/sh: command substitution: line 1: `comm -13 <(git branch -a | sed -e 's,.*[ /],,' | sort | uniq) <(ls /tmp/ghpages1548 | sed -e 's,.*/,,')'
cp draft-ietf-core-object-security.html index.html
cp -f index.html /tmp/ghpages1548/index.html
cp: cannot create regular file ‘/tmp/ghpages1548/index.html’: No such file or directory
make: *** [/tmp/ghpages1548/index.html] Fel 1

Any help is much appreciated, as usual! :)

Francesca

Trailing whitespace check breaks files

While I hate trailing whitespace as much as the next guy, preventing checkin when people use an editor that generates it is pretty harsh. (The easy solution is to switch to DOS line ends, in which case the check doesn't work any more :-).

The wholesale removal of whitespace from existing files that this triggers is going to cloud the history.

(Oh, and the use of trailing whitespace as a format effector actually is a markdown feature. Bleh, I know.)

Optimization for CI: use `kdrfc -r`

(Please read this after the I-D deadline.)

After having waited waiting for the umpteenth recompilation of Python lxml:

Maybe use kdrfc -r to build from markdown (kramdown flavor), which uses the tools server instance of xml2rfc instead of a locally installed one.

Pro:

• Doesn't need an xml2rfc installation, which needs about 5 CPU seconds and dominates the wait for CI builds¹.
• For local processing, make sure the most recent version of xml2rfc is actually being used.

Con:

• Relies on another point of failure (tools server).
• Error messages are slightly budgered by the tools server, but still clearly visible.
• Warning messages from xml2rfc are invisible unless they are accompanied by an error.

Footnotes

1. (followed by the refcache fills; I need to figure out a way to parallelize those) ↩

"Setup repository" commit: reference tool used

Please consider adding a note to the "Setup repository" commit like this:

Setup repository for draft-...

Structure created from the i-d-template repository.

This would help people new to the i-d-template mechanisms discover it in the first place when they set up a draft in the style of other documents in their working group.

Offline Support

One of the great things about Git is that repos are mostly autonomous -- you can commit, switch branches, etc. without network access.

Unfortunately, Kramdown attempts to fetch references every time, even if they're already downloaded. Is there a way we can mute this, so that make (and therefore, commit with the pre-commit hook) can run successfully without being able to refresh the references?

sed: 1: "/<title[^>]*>/,/<\/titl ...": extra characters at the end of q command

On macOS Sierra, make -f lib/setup.mk fails in an interesting way, leaving an empty index.html:

DRAFT_NAME=$(echo draft-ietf-lwig-coap.xml | cut -f 1 -d . -); \
	  AUTHOR_LABEL=$(echo draft-ietf-lwig-coap.xml | cut -f 2 -d - -); \
	  WG_NAME=$(echo draft-ietf-lwig-coap.xml | cut -f 3 -d - -); \
	  DRAFT_STATUS=$(test "$AUTHOR_LABEL" = ietf && echo Working Group || echo Individual); \
	  GITHUB_USER=lwig-wg; GITHUB_REPO=coap; \
	  DRAFT_TITLE=$(sed -e '/<title[^>]*>/,/<\/title>/{s/.*<title[^>]*>//;/<\/title>/{s/<\/title>.*//;H;x;q};H;};d' draft-ietf-lwig-coap.xml | xargs echo); \
	  sed -i~ -e 's~{DRAFT_NAME}~'"$DRAFT_NAME"'~g' -e 's~{DRAFT_TITLE}~'"$DRAFT_TITLE"'~g' -e 's~{DRAFT_STATUS}~'"$DRAFT_STATUS"'~g' -e 's~{GITHUB_USER}~'"$GITHUB_USER"'~g' -e 's~{GITHUB_REPO}~'"$GITHUB_REPO"'~g' -e 's~{WG_NAME}~'"$WG_NAME"'~g' README.md CONTRIBUTING.md LICENSE.md
sed: 1: "/<title[^>]*>/,/<\/titl ...": extra characters at the end of q command

(No, I don't understand this error message. But then I'm not an sed expert...)
Hadn't noticed this so far since the travis version seems to work.

Add "fork me on github" to the index.html version

draft-*.html and index.html are currently the same. Use code from here:

https://github.com/blog/273-github-ribbons

to add a fork-me button on index.html.

Problems with setup

Hello,

First of all thank you so much for this work!
About the issue, I just wanted to say that I set up a repository in September, and that at the time I didn't have any problems. Now I tried again, and this time I'm running on some issues. When I run: "make -f lib/setup.mk":

$ make -f lib/setup.mk
lib/main.mk:91: .targets: No such file or directory
lib/setup.mk:44: lib/.template-files.mk: No such file or directory
lib/setup.mk:30: *** You have uncommitted changes, please commit them before running setup. Stop.

And of course at this point a git status shows that .targets and lib/ are untracked.
I go on and add them to the .gitignore, after which the setup goes on ok.
Next problem is with "make ghpages"

$ make ghpages
lib/main.mk:91: .targets: No such file or directory
cp draft-ietf-core-object-security.html index.html
mkdir /tmp/ghpages9540
cp -f index.html draft-ietf-core-object-security.html draft-ietf-core-object-security.txt /tmp/ghpages9540
git clean -qfdX
git checkout gh-pages
error: The following untracked working tree files would be overwritten by checkout:
index.html
Please move or remove them before you can switch branches.
Aborting
make: *** [ghpages] Error 1

Removing doesn't really help of course. since index is copied in the beginning.

Am I doing something wrong?

Add setup target

The makefile includes good rules for setting up gh-pages, but the readme files are not so good. Most of the information can be retrieved from the draft(s). A second setup-readme target might be added to rename README-template.md to README.md and to add all the information that can be retrieved.

"No file named draft-*.xml" with -00 draft filename

I've just set up a repository using this template. When I named my draft draft-jordan-foobar-00.xml, make -f lib/setup.mk threw an error about it not being able to find the file, but when I renamed it to draft-jordan-foobar-latest.xml everything worked. Seems like this is mostly a documentation problem?

Build a proper CI system

This is a non-trivial system, tests for the following would be nice:

1. setup process
2. building the different variants
3. using CI

md issues or me?

Martin - why am I being a n00b?! I chose md, I ran make, I got files, and a gh-pages branch. But nothing is rendering on index.html on ghpages. I redid it with xml suspecting a md weirdness, I'm still getting nothing. Boo! If you can help that would be fab - otherwise I'll just keep adding to the xml and hoping I can solve it later!

https://github.com/ietf-slim-wg/slim-use-cases

Note branch versions in gh-pages

The commit messages in gh-pages mention only a single commit hash (presumably of master), but take input from all branches.

I'd suggest that a line be added for each built branch and their respective commit hashes, for otherwise that information can only be obtained by checking which changes have been incorporated or by making assumptions about which branches were pushed to the commit server at build-time.

(From my PoV, please consider this low priority; it'd be a nice addition but most of the time the assumption that everything is in the latest state holds anyway.)

error while make

Make ghpages work for me on 5/18, but now after "make ghpages" I get:

      mkdir /tmp/ghpages808
      cp -f index.html draft-turner-ccmib.html draft-turner-ccmib.txt /tmp/ghpages808 /tmp/ghpages808
      cp: /tmp/ghpages808 is a directory (not copied).
      make: *** [lib/ghpages.mk:56: ghpages] Error 1
      rm /tmp/ghpages808make: unlink: /tmp/ghpages808: Operation not permitted

why make version 4

v 3.81 is bundled with my ubuntu system. is v4 really needed?

No rule to make targetr `Makefile'

; ls
LICENSE  README.md  draft-rsalz-drbg-speck-wap-wep.md  lib
; cat .gitmodules
[submodule "lib"]
        path = lib
        url = https://github.com/martinthomson/i-d-template
; git -C lib pull origin master
From https://github.com/martinthomson/i-d-template
 * branch            master     -> FETCH_HEAD
Already up-to-date.
; make -f lib/setup.mk
make: *** No rule to make target `Makefile', needed by `setup-readme'.  Stop.
        exit 2
;

make gh-pages behaving incorrectly

If you look at the following link https://github.com/core-wg/oscoap/tree/gh-pages you will see that there is a large number of directories that have been created. These are being created when a merge is being done directly on the project.

Link to a travis build is https://travis-ci.org/core-wg/oscoap/builds/214279498

Branch links

The gh-pages branch contains builds for every branch, but the "Editor's copy" and "Compare Working group..." links always show the master version, even if the user is browsing them from a branch.

I'd like to see the builds of the branches linked too, but changing README.md for every branch is not an option to me.

The solution I'd propose is to make the build process generate a /+branches/index.html file, which contains "branch copy", "compare WG draft to branch" and "compare master to branch" links for each branch.

With that design, it'd be even an option that /+branches/ be instructed to inspect the referrer for a branch guess, so we could have links like "Editor's copy (→ github.io/.../), Editor branch (→ github.io/draft-name/+branches/#go-to-branch)" that javascript-redirect to github.io/draft-name// or the #go-to-diff that redirect to an appropriate rfcdiff address, but that could be a second step.

Would you consider adding that, and/or would you accept a PR if I find time to do it myself?

Feature request: Put the output of "make diff" into ghpages

Would be nice to have, in addition to the official/editor's links, an easy-to-parse diff between them. You already have "make diff", so it seem like it would "just" be a matter of creating the diff, stuffing it into the ghpages output, and adding the links to the index files you create.

(If you don't get to it, I might see if I can decode your Makefiles at some point over the holiday.)

issues aren't saved properly

See httpwg/http-extensions@b69b534

index.html generation (for multiple documents) on Travis or OS X

The "define INDEX_HTML =" part of ghpages.mk does not seem to work with Travis' make. It appears to leave an empty variable, so we get (https://travis-ci.org/core-wg/yang-cbor) something like

echo -e '' >index.html

Workaround: make ghpages and manually push the gh-pages branch on the laptop.

Also, neither Travis' nor OS X's echo as called from make supports -e, so the -e option gets dumped literally, as can be seen in https://core-wg.github.io/yang-cbor/ (more of an esthetic problem here). (The result of both problems is that for Travis, the entire contents of index.html will be -e.)

CI_USER and CI_REPO initialized incorrectly on Travis

From the gmake info file:

'$(patsubst PATTERN,REPLACEMENT,TEXT)'
     Finds whitespace-separated words in TEXT that match PATTERN and
     replaces them with REPLACEMENT.  

Apparently, id.mk tries to use a pattern that is not matching a whitespace-separated word.

See https://travis-ci.org/core-wg/yang-cbor -- or try:

TRAVIS_REPO_SLUG = bar/baz

CI_USER = $(patsubst /%,,$(TRAVIS_REPO_SLUG))$(CIRCLE_PROJECT_USERNAME)
CI_REPO = $(patsubst %/,,$(TRAVIS_REPO_SLUG))$(CIRCLE_PROJECT_REPONAME)
$(warning CI_USER //$(CI_USER)//)
$(warning CI_REPO //$(CI_REPO)//)

FIXED_CI_USER = $(subst /,,$(dir $(TRAVIS_REPO_SLUG)))
FIXED_CI_REPO = $(notdir $(TRAVIS_REPO_SLUG))
$(warning FIXED_CI_USER //$(FIXED_CI_USER)//)
$(warning FIXED_CI_REPO //$(FIXED_CI_REPO)//)

Proposed fix: see above.

What's the problem with Xcode's make?

I filed this PR on ietf-wg-acme to fix an XCode/Xcode typo, and they asked me to file it upstream. Upstream, in f746176, the typo now appears in an advisory to install Homebrew's version of make, and an assertion about Xcode's version being "subtly broken".

What sort of subtle brokenness is at issue? Is there an openradar for the problem? Thanks.

"make diff" shows the wrong draft numbers

The "-latest" to "-nn" transformation is applied to the current version, not the previous version. This results in -00 versus -latest comparisons appearing to be -latest versus -01.

wrong FETCH_HEAD when using i-d-template as named submodule

After having passed the --name i-d-template parameter to git submodule add, the FETCH_HEAD in L7 of lib/update.mk is not recognized any more (because now it's .git/modules/i-d-template/FETCH_HEAD instead of `.git/modules/$(LIBDIR)/FETCH_HEAD).

$ cat .gitmodules 
[submodule "i-d-template"]
    path = lib
    url = https://github.com/martinthomson/i-d-template
$ ls .git/modules/
i-d-template

Unfortunately, I am not sure how to fix this properly (maybe by saying "Don't use --name" in the README).

Bad link to more detailed guide in README.md

https://github.com/martinthomson/blob/master/doc/WG-SETUP.md gives a 404

draft-ietf....md in .gitignore?

This is just confusing and not causing a problem:

In several of my repos the .md is in the .gitignore.
I didn't put it there, and the file is tracked properly anyway...