sethen / markdown-include

:page_with_curl: Include markdown files into other markdown files with C style syntax

Table of Contents

• markdown-include
  • Compile your markdown files
  • Make a table of contents
• How To Install
• How To Use From The Command Line
  • markdown.json
• How To Use As A Module
  • API
    • buildLink
    • buildLinkString
    • compileFiles
    • compileHeadingTags
    • findHeadingTags
    • findIncludeTags
    • parseHeadingTag
    • parseIncludeTag
    • processFile
    • processIncludeTags
    • replaceIncludeTags
    • replaceWith
    • resolveCustomTags
    • stripTag
    • stripTagsInFile
    • writeFile
• How To Make Plugins
  • Tutorial
• How It Works
• License

markdown-include

markdown-include is built using Node.js and allows you to include markdown files into other markdown files using a C style include syntax.

Compile your markdown files

markdown-include's main feature is that it allows you to include markdown files into other markdown files. For example, you could place the following into a markdown file:

#include "markdown-file.md"
#include "another-markdown-file.md"

And assuming that markdown.file.md contents are:

Something in markdown file!

And assuming that another-markdown-file.md contents are:

Something in another markdown file!

It would compile to:

Something in markdown file!
Something in another markdown file!

Pretty neat, huh?

Make a table of contents

Aside from compiling your markdown files, markdown-include can also build your table of contents. This works by evaluating the heading tags inside of your files. Since markdown works on using # for making HTML headings, this makes it easy to assemble table of contents from them. The more # you have in front of your headings (up to 6) will decide how the table of contents is built. Use one # and it's a top level navigation item... Use two # and it would be underneath the previous navigation item and so on.

For each heading that you would like to be included in a table of contents just add !heading to the end of it.

How To Install

markdown-include is available on npm for easy installation:

npm install markdown-include

Use the -g flag if you wish to install markdown-include globally on your system. Use the --save flag to save in your package.json file in your local project.

How To Use From The Command Line

markdown-include is very easy to use whether on the command line or in your own node project. Each can help you compile your markdown files as you see fit. markdown-include does require that you define a markdown.json file with your options for compile. See below for all of the options available to you.

Run from the command line to compile your documents like so:

node_modules/bin/cli.js path/to/markdown.json

markdown.json

markdown.json can be populated with the following options:

Option	Type	Description
build	String	File path of where everything should be compiled, like README.md.
files	Array	Array of files to to compile.
tableOfContents	Object	Object to hold options for table of contents generation.
tableOfContents.heading	String	Heading for table of contents (use markdown syntax if desired).
tableOfContents.lead	String	What navigation items in table of contents lead with. If value is number will add numbers before each item and subitem. If not, will add asterisks. Refer to markdown syntax to understand the difference.

How To Use As A Module

Just require in your node project:

var markdownInclude = require('markdown-include');

From there, you can use markdown-include's API to fit your needs.

API

When using as a module, markdown-include offers an API for you to work with markdown files as detailed below:

---

buildLink

Description

A method for making markdown style anchor tags.

Signature

buildLink(title: String, anchor: String) => String

Parameters

Parameter(s)	Type	Description
title	String	Title of markdown style link.
anchor	String	Markdown style anchor for linking.

Example

var markdownInclude = require('markdown-include');
markdownInclude.buildLink("My Link String", "#my-link-string"); // [My Link String](#my-link-string)

---

buildLinkString

Description

A method for taking strings and building friendly markdown links. This is mostly used internally for building the table of contents.

Signature

buildLinkString(str: String) => String

Parameters

Parameter(s)	Type	Description
str	String	File path of where everything should be compiled, like README.md.

Example

var markdownInclude = require('markdown-include');
markdownInclude.buildLinkString("My Link String"); // my-link-string

---

compileFiles

Description

This is probably the most important method in markdown-include. It takes a path to your markdown.json file, reads your options and returns a promise. When the promise is resolved it returns the data to you. This is exactly the same as running markdown-include in the command line as it runs through the whole lifecycle.

Signature

compileFiles(path: String) => Object<Promise>

Parameters

Parameter(s)	Type	Description
path	String	Compiles files when given the path to markdown.json

Example

var markdownInclude = require('markdown-include');
markdownInclude.compileFiles("path/to/markdown.json").then(function (data) {
	// do something with compiled files
});

---

compileHeadingTags

Description

A method for compiling heading tags (!heading) in a given file.

Signature

compileHeadingTags(file: String)

Parameters

Parameter(s)	Type	Description
file	String	File with !heading tags

Example

var markdownInclude = require('markdown-include');
markdownInclude.compileHeadingTags("my_file.md");

---

findHeadingTags

Description

A method for finding heading tags (!heading) in a string.

Signature

findHeadingTags(data: String) => Array<String>

Parameters

Parameter(s)	Type	Description
data	String	Data with !heading tags

Example

var markdownInclude = require('markdown-include');
markdownInclude.findHeadingTags("### A Heading !heading"); // [ "### A Heading !heading" ]

---

findIncludeTags

Description

A method for finding include tags (#include "my-include.md") in a string.

Signature

findIncludeTags(data: String) => Array<String>

Parameters

Parameter(s)	Type	Description
data	String	Data with #include "my-include.md" tags

Example

var markdownInclude = require('markdown-include');
markdownInclude.findIncludeTags('#include "my-include.md"'); // [ '#include "my-include.md"' ]

---

parseHeadingTag

Description

Parses a heading tag based on the amount of asterisks present before it (### Heading)

Signature

parseHeadingTag(headingTag: String) => Object<count: Number, headingTag: String>

Parameters

Parameter(s)	Type	Description
headingTag	String	Heading tag to parse

Example

var markdownInclude = require('markdown-include');
markdownInclude.parseHeadingTag('### Heading'); // { count: 3, headingTag: 'Heading' }

---

parseIncludeTag

Description

Parses a include tag (#include "my-include.md")

Signature

parseIncludeTag(tag: String) => String

Parameters

Parameter(s)	Type	Description
tag	String	Heading tag to parse

Example

var markdownInclude = require('markdown-include');
markdownInclude.parseIncludeTag('#include "my-include.md"'); // "my-include.md"

---

processFile

Description

Processes a file and adds it to the build object for compiling.

Signature

processFile(file: String, currentFile: String)

Parameters

Parameter(s)	Type	Description
file	String	File for processing
currentFile	String	Current file include file was found in for additional processing

Example

var markdownInclude = require('markdown-include');
markdownInclude.processFile('another-include.md', 'my-include.md');

---

processIncludeTags

Description

Processes a file and adds it to the build object for compiling.

Signature

processIncludeTags(file: String, currentFile: String, tags: Array<String>) => Array<String>

Parameters

Parameter(s)	Type	Description
file	String	File for processing
currentFile	String	Current file include file was found in for additional processing
tags	Array<String>	Current file include file was found in for additional processing

Example

var markdownInclude = require('markdown-include');
markdownInclude.processIncludeTags('another-include.md', 'my-include.md', ['one-include.md']); // ['one-include.md']

---

replaceIncludeTags

Description

Replaces include tags in given file with actual data (file must be added to build object first with processFile).

Signature

replaceIncludeTags(file: String) => String

Parameters

Parameter(s)	Type	Description
file	String	File to replace include tags

Example

var markdownInclude = require('markdown-include');
markdownInclude.replaceIncludeTags('my-include.md'); // 'Content in my-include.md!'

---

replaceWith

Description

Utility method for transforming a string.

Signature

replaceWith(Object<string: String, index: Number, preserve: Boolean, replacement: String>) => String

Parameters

Parameter(s)	Type	Description
obj.string	String	String to transform
obj.index	Number	Index to start transformation to end
obj.preserve	Boolean	Preserve original string
obj.replacement	String	String to use for replacement

Example

var markdownInclude = require('markdown-include');
markdownInclude.replaceWith({
	string: "string",
	index: 4,
	replacement: "myString"
}); // 'somemyString'

var markdownInclude = require('markdown-include');
markdownInclude.replaceWith({
	string: "string",
	index: 4,
	preserve: true,
	replacement: "myString"
}); // 'somemyStringthing'

---

resolveCustomTags

Description

Method for resolving custom tags. Looks in customTags object attached to markdown include module.

Signature

resolveCustomTags(data: String) => String

Parameters

Parameter(s)	Type	Description
data	String	String with custom tags

Example

var markdownInclude = require('markdown-include');
markdownInclude.resolveCustomTags('### Custom Tag !myCustomTag');

---

stripTag

Description

Method for stripping tags in a string.

Signature

stripTag(Object<tag: String, pattern: String>) => String

Parameters

Parameter(s)	Type	Description
obj.tag	String	String with tag in it
obj.pattern	String	Pattern to replace in tag

Example

var markdownInclude = require('markdown-include');
markdownInclude.stripTag('### Custom Tag !myCustomTag'); // ### Custom Tag

---

stripTagsInFile

Description

Strips tags in a given file.

Signature

stripTag(Object<data: String, pattern: String, string: String, replace: String|Function>) => String

Parameters

Parameter(s)	Type	Description
obj.data	String	Data with tags to strip
obj.pattern	String	Pattern to look for
obj.string	String	String to replace
obj.replace	`String	Function`

Example

var markdownInclude = require('markdown-include');
markdownInclude.stripTagsInFile({
	data: 'String with tags !ignore',
	pattern: <RegExp>,
	string: '!ignore'
});

---

writeFile

Description

Writing contents to a file using the file path outlined in markdown.json.

Signature

stripTag(parsedData: String) => Object<Promise>

Parameters

Parameter(s)	Type	Description
parsedData	String	String to write

Example

var markdownInclude = require('markdown-include');
markdownInclude.writeFile('contents').then(function (data) {
	// continue...
});

---

How To Make Plugins

Plugins are now supported as of 0.4.0 of markdown-include. Adding plugins to markdown-include to facilitate the transformation of custom tags is quite trivial.

Plugins are best used when markdown-include is being required as a module. If you wish to make this available via the command line, you must require markdown-include in a node module and call it from the command line.

Tutorial

Let's pretend we want to add a custom tag called !myTag that follows the pattern of #phrase !myTag. All we need to do is register the plugin with markdown-include

First, require markdown-include:

var markdownInclude = require('markdown-include');

Second, register your plugin with with your desired pattern to match and desired replacement. You can replace your tag with another string to do your desired work:

var markdownInclude = require('markdown-include');
markdownInclude.registerPlugin({
	pattern: /^#.+ !myTag/gm,
	replace: 'myString!'
});

In the example above, we're just replacing our tag with a string. If you would rather use a function, you can do this like so (you must return a value to replace with):

var markdownInclude = require('markdown-include');
markdownInclude.registerPlugin({
	pattern: /^#.+ !myTag/gm,
	replace: function (tag) {
		// do something with tag...
		return 'myString!'
	}
});

pattern is the regular expression that should be looked for. replace is your desired replacement for the tag once it's found.

This gives you free range to do whatever you want with the tag you want to replace. Once the tag is encountered markdown-include will run the function.

After the tag and it's replacement is registered, it's business as usual:

markdownInclude.compileFiles('../path/to/markdown.json').then(function () {
	// do something after compiling
});

You can also use another form of registering a plugins if it fits your coding style better:

markdownInclude.registerPlugin(/^#.+ !myTag/gm, function (tag) {
	return 'my replacement!';
});

How It Works

markdown-include works by recursively going through files based on the tags that are found. For instance, consider the following in a _README.md file:

#include "first-file.md"

Let's also consider that first-file.md contains the following:

#include "third-file.md"

Let's also consider that markdown.json contains the following:

{
	"build" : "README.md",
	"files" : ["_README.md"]
}

markdown-include will first read the contents of _README.md and look for include tags. It will find #include "first-file.md" first. From there it will parse the tag, open first-fild.md and find include tags in that file. This process continues until no more include tags are found.

At that point it will start over in the original file and parse other include tags if they exist. Along the way, markdown-include will parse each file and keep a record of the contents. Once the process is finished, a file will be written in README.md with all of the compiled content.

As you can see, you only need to reference one file which would be _README.md. We didn't need to add first-file.md or third-file.md... markdown-include does that compiling for us by making an internal chain.

NOTE: You must provide markdown-include with the entire file path you're trying to find in your working directory. For example, if first-file.md and third-file.md were in the docs directory together and first-file.md was trying to include third-file.md you would need to do the following in first-file.md:

#include "docs/third-file.md"

This is because markdown-include doesn't make any assumptions about where your files are. Use the correct paths or you could run into errors!

License

ISC License

Copyright (c) 2015-2016, Sethen Maleno

Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

markdown-include's Issues

Includes with filenames that include "." are not recognized as includes

If a filename includes a "." (separate from the ".md" extension), the file is not found/included and the string is left as-is.

//_readme.md

# A heading

Some text

#include "path/to/working/include.md"

More text

#include "path/to/other/include.5.md"

Resulting markdown:

# A heading

Some text

Some working markdown include

More text

#include "path/to/other/include.5.md"

& characters can not be parsed

This breaks the parse if & characters are in the title for the table of contents... Will have to follow Github guidelines on this.

Adding License file / Hint in Readme

A simple Suggestion,

After reading package.json, found the code is released under ISC License.

It would be nice if a separate licence file in repo and a hint at bottom of read file.

If acceptable, could make a PR 😄

Good Work <3

Thanks

ES6 overhaul

The current code is written in ES5. The code should be overhauled with ES6 using Webpack or a similar compiler.

Duplicate sub headings lead to first link

If you have duplicate sub headings the link generated for them in the table of contents will just lead to the first match it finds. For instance, if you had:

# A Heading !heading
## A Subheading !heading

# Another Heading !heading
## A Subheading !heading

The table of contents generates the link based on the heading text. So both links for A Subheading would be the first subheading content.

Table of contents not generated correctly if level 1 or 2 headings not included

If only level 3 or lower headings are included (using the !heading marker), the table of contents will be indented too much in the generated Markdown and the resulting HTML will not be a list.

This source file:

# Title
## Section
### Heading !heading
#### Subheading 1 !heading
#### Subheading 2 !heading

will result in this table of contents:

␣␣␣␣* [Heading](#heading)
␣␣␣␣␣␣* [Subheading 1](#subheading-1)
␣␣␣␣␣␣* [Subheading 2](#subheading-2)

(with the spaces represented by ␣)

TypeScript overhaul

markdown-include should move to using TypeScript.

How do I use this on GitHub?

Duplicate TOC entries when compiling MD files from several JSONs in single JS

fs.readdirSync("./").filter(f => f.toLowerCase().endsWith(".json")).forEach(function(markdownJson) {
  markdownInclude.compileFiles(markdownJson).then(function () {
    console.info(markdownJson + ' have been built successfully');
  })
});

Example:
There are 3 JSON files in the directory with different files for compilation. But some of the MD files are reused in different compiled MD files.

1st.md content

#include my.md

2nd.md content

#include my.md
#include my2.md

3rd.md content

#include my.md
#include my3.md

my2.md content

some content about my2.
#include my3.md

every MD file has it's own unique Heading with !heading tag for inclusion into TOC.

Create build path if not existing

If I specify build path to be 'folder-not-existed-yet/doc.md', the script will fail. I don't think this is a commonly expected behavior of a CLI tool though. As far as I've seen most build tools will create the build path if not existing already.

, characters can't be parsed

Using , character in headings that are built in the table of contents break the links.

Table of Contents broken on special characters

At least on GitHub... I'm autogenerating the TOC and it thinks that the headings will include special characters, but GitHub strips them out. At a minimum, the following characters have broken the links my documentation:

• (
• )
• `
• .
• ,
• %

I think all of them should just be dropped, but I'm not 100% sure.

Excerpts?

Great module. I dunno if this is the venue for this, but I haven't found another module that can do excerpts. For the purposes of this issue, I'm defining an excerpt as an include+delimiter. Seems fairly straightforward.

Any interest in that type of feature? I'd be happy to contribute if so.

Code inclusion

Currently, markdown-include can pull in any type of file... It just looks at the path and goes from there. It should be smart enough to know when it's including a code snippet and wrap it with the appropriate code block for easier documentation. For instance, you could do

#include "ruby.rb"

And markdown-include should compile

class MyClass {
  def initialize(arg) {
     @arg = arg;
  }
}

wrapped in appropriate backticks that correspond to the language for syntax highlighting if available.

Add possibility to customize tag used to include heading in TOC.

It would be nice if there was possibility to customize tag (I mean !heading) which tells whether specified heading is included in table of contents.

I have a case, when I compile a few markdown files into a single file and I'd like to have a TOC in resulting file, but at the same time I'm using the original file and I don't want !heading text to be visible in headings. With customization I could put tag inside comment and specify that TOC should contain headings marked with, for instance, <!-- !heading -->.

Add back to top link

Currently, no back to top link exists with generated content. When content is generated, the user should have the option of adding one.

Dry mode?

Having a dry mode for CI would be great 🙏

Using in Browser

Hi, I was really hoping to use this in the browser. Is there any way it can be adapted for that? I'm already using several other markdown plugins client-side, so I hoped this would simply work.

No new lines at the end of markdown files breaks parser

If no new lines are present at the bottom of individual markdown files, the parser will get confused and just concatenate them as they are. This breaks titles that may be present in other files.