Autolink fails for URLs with >3 parts in the domain

lezer-markdown

This is an incremental Markdown (CommonMark with support for extension) parser that integrates well with the Lezer parser system. It does not in fact use the Lezer runtime (that runs LR parsers, and Markdown can't really be parsed that way), but it produces Lezer-style compact syntax trees and consumes fragments of such trees for its incremental parsing.

Note that this only parses the document, producing a data structure that represents its syntactic form, and doesn't help with outputting HTML. Also, in order to be single-pass and incremental, it doesn't do some things that a conforming CommonMark parser is expected to do—specifically, it doesn't validate link references, so it'll parse [a][b] and similar as a link, even if no [b] reference is declared.

The @codemirror/lang-markdown package integrates this parser with CodeMirror to provide Markdown editor support.

The code is licensed under an MIT license.

Interface

parser: MarkdownParser: The default CommonMark parser.

class MarkdownParser extends Parser

A Markdown parser configuration.

nodeSet: NodeSet: The parser's syntax node types.
configure(spec: MarkdownExtension) → MarkdownParser: Reconfigure the parser.
parseInline(text: string, offset: number) → Element[]: Parse the given piece of inline text at the given offset, returning an array of Element objects representing the inline content.

interface MarkdownConfig

Objects of this type are used to configure the Markdown parser.

props⁠?: readonly NodePropSource[]: Node props to add to the parser's node set.
defineNodes⁠?: readonly (string | NodeSpec)[]: Define new node types for use in parser extensions.
parseBlock⁠?: readonly BlockParser[]: Define additional block parsing logic.
parseInline⁠?: readonly InlineParser[]: Define new inline parsing logic.
remove⁠?: readonly string[]: Remove the named parsers from the configuration.
wrap⁠?: ParseWrapper: Add a parse wrapper (such as a mixed-language parser) to this parser.

type MarkdownExtension = MarkdownConfig | readonly MarkdownExtension[]: To make it possible to group extensions together into bigger extensions (such as the Github-flavored Markdown extension), reconfiguration accepts nested arrays of config objects.

parseCode(config: Object) → MarkdownExtension

Create a Markdown extension to enable nested parsing on code blocks and/or embedded HTML.

config

codeParser⁠?: fn(info: string) → Parser | null: When provided, this will be used to parse the content of code blocks. info is the string after the opening ``` marker, or the empty string if there is no such info or this is an indented code block. If there is a parser available for the code, it should return a function that can construct the parse.
htmlParser⁠?: Parser: The parser used to parse HTML tags (both block and inline).

GitHub Flavored Markdown

GFM: MarkdownConfig[]: Extension bundle containing Table, TaskList, Strikethrough, and Autolink.

Table: MarkdownConfig

This extension provides GFM-style tables, using syntax like this:

| head 1 | head 2 |
| ---    | ---    |
| cell 1 | cell 2 |

TaskList: MarkdownConfig: Extension providing GFM-style task list items, where list items can be prefixed with [ ] or [x] to add a checkbox.

Strikethrough: MarkdownConfig: An extension that implements GFM-style Strikethrough syntax using ~~ delimiters.

Autolink: MarkdownConfig: Extension that implements autolinking for www./http:///https:///mailto:/xmpp: URLs and email addresses.

Other extensions

Subscript: MarkdownConfig: Extension providing Pandoc-style subscript using ~ markers.

Superscript: MarkdownConfig: Extension providing Pandoc-style superscript using ^ markers.

Emoji: MarkdownConfig: Extension that parses two colons with only letters, underscores, and numbers between them as Emoji nodes.

Extension

The parser can, to a certain extent, be extended to handle additional syntax.

interface NodeSpec

Used in the configuration to define new syntax node types.

name: string: The node's name.
block⁠?: boolean: Should be set to true if this type represents a block node.
composite⁠?: fn(cx: BlockContext, line: Line, value: number) → boolean: If this is a composite block, this should hold a function that, at the start of a new line where that block is active, checks whether the composite block should continue (return value) and optionally adjusts the line's base position and registers nodes for any markers involved in the block's syntax.
style⁠?: Tag | readonly Tag[] | Object<Tag | readonly Tag[]>: Add highlighting tag information for this node. The value of this property may either by a tag or array of tags to assign directly to this node, or an object in the style of styleTags's argument to assign more complicated rules.

class BlockContext implements PartialParse

Block-level parsing functions get access to this context object.

lineStart: number: The start of the current line.
parser: MarkdownParser: The parser configuration used.
depth: number: The number of parent blocks surrounding the current block.
parentType(depth⁠?: number = this.depth - 1) → NodeType: Get the type of the parent block at the given depth. When no depth is passed, return the type of the innermost parent.
nextLine() → boolean: Move to the next input line. This should only be called by (non-composite) block parsers that consume the line directly, or leaf block parser nextLine methods when they consume the current line (and return true).
prevLineEnd() → number: The end position of the previous line.
startComposite(type: string, start: number, value⁠?: number = 0): Start a composite block. Should only be called from block parser functions that return null.
addElement(elt: Element): Add a block element. Can be called by block parsers.
addLeafElement(leaf: LeafBlock, elt: Element): Add a block element from a leaf parser. This makes sure any extra composite block markup (such as blockquote markers) inside the block are also added to the syntax tree.
elt(type: string, from: number, to: number, children⁠?: readonly Element[]) → Element elt(tree: Tree, at: number) → Element: Create an Element object to represent some syntax node.

interface BlockParser

Block parsers handle block-level structure. There are three general types of block parsers:

Composite block parsers, which handle things like lists and blockquotes. These define a parse method that starts a composite block and returns null when it recognizes its syntax.
Eager leaf block parsers, used for things like code or HTML blocks. These can unambiguously recognize their content from its first line. They define a parse method that, if it recognizes the construct, moves the current line forward to the line beyond the end of the block, add a syntax node for the block, and return true.
Leaf block parsers that observe a paragraph-like construct as it comes in, and optionally decide to handle it at some point. This is used for "setext" (underlined) headings and link references. These define a leaf method that checks the first line of the block and returns a LeafBlockParser object if it wants to observe that block.

name: string: The name of the parser. Can be used by other block parsers to specify precedence.
parse⁠?: fn(cx: BlockContext, line: Line) → boolean | null: The eager parse function, which can look at the block's first line and return false to do nothing, true if it has parsed (and moved past a block), or null if it has started a composite block.
leaf⁠?: fn(cx: BlockContext, leaf: LeafBlock) → LeafBlockParser | null: A leaf parse function. If no regular parse functions match for a given line, its content will be accumulated for a paragraph-style block. This method can return an object that overrides that style of parsing in some situations.
endLeaf⁠?: fn(cx: BlockContext, line: Line, leaf: LeafBlock) → boolean: Some constructs, such as code blocks or newly started blockquotes, can interrupt paragraphs even without a blank line. If your construct can do this, provide a predicate here that recognizes lines that should end a paragraph (or other non-eager leaf block).
before⁠?: string: When given, this parser will be installed directly before the block parser with the given name. The default configuration defines block parsers with names LinkReference, IndentedCode, FencedCode, Blockquote, HorizontalRule, BulletList, OrderedList, ATXHeading, HTMLBlock, and SetextHeading.
after⁠?: string: When given, the parser will be installed directly after the parser with the given name.

interface LeafBlockParser

Objects that are used to override paragraph-style blocks should conform to this interface.

nextLine(cx: BlockContext, line: Line, leaf: LeafBlock) → boolean: Update the parser's state for the next line, and optionally finish the block. This is not called for the first line (the object is contructed at that line), but for any further lines. When it returns true, the block is finished. It is okay for the function to consume the current line or any subsequent lines when returning true.
finish(cx: BlockContext, leaf: LeafBlock) → boolean: Called when the block is finished by external circumstances (such as a blank line or the start of another construct). If this parser can handle the block up to its current position, it should finish the block and return true.

class Line

Data structure used during block-level per-line parsing.

text: string: The line's full text.
baseIndent: number: The base indent provided by the composite contexts (that have been handled so far).
basePos: number: The string position corresponding to the base indent.
pos: number: The position of the next non-whitespace character beyond any list, blockquote, or other composite block markers.
indent: number: The column of the next non-whitespace character.
next: number: The character code of the character after pos.
skipSpace(from: number) → number: Skip whitespace after the given position, return the position of the next non-space character or the end of the line if there's only space after from.
moveBase(to: number): Move the line's base position forward to the given position. This should only be called by composite block parsers or markup skipping functions.
moveBaseColumn(indent: number): Move the line's base position forward to the given column.
addMarker(elt: Element): Store a composite-block-level marker. Should be called from markup skipping functions when they consume any non-whitespace characters.
countIndent(to: number, from⁠?: number = 0, indent⁠?: number = 0) → number: Find the column position at to, optionally starting at a given position and column.
findColumn(goal: number) → number: Find the position corresponding to the given column.

class LeafBlock

Data structure used to accumulate a block's content during leaf block parsing.

parsers: LeafBlockParser[]: The block parsers active for this block.
start: number: The start position of the block.
content: string: The block's text content.

class InlineContext

Inline parsing functions get access to this context, and use it to read the content and emit syntax nodes.

parser: MarkdownParser: The parser that is being used.
text: string: The text of this inline section.
offset: number: The starting offset of the section in the document.
char(pos: number) → number: Get the character code at the given (document-relative) position.
end: number: The position of the end of this inline section.
slice(from: number, to: number) → string: Get a substring of this inline section. Again uses document-relative positions.
addDelimiter(type: DelimiterType, from: number, to: number, open: boolean, close: boolean) → number: Add a delimiter at this given position. open and close indicate whether this delimiter is opening, closing, or both. Returns the end of the delimiter, for convenient returning from parse functions.
addElement(elt: Element) → number: Add an inline element. Returns the end of the element.
findOpeningDelimiter(type: DelimiterType) → number | null: Find an opening delimiter of the given type. Returns null if no delimiter is found, or an index that can be passed to takeContent otherwise.
takeContent(startIndex: number) → Element[]: Remove all inline elements and delimiters starting from the given index (which you should get from findOpeningDelimiter, resolve delimiters inside of them, and return them as an array of elements.
skipSpace(from: number) → number: Skip space after the given (document) position, returning either the position of the next non-space character or the end of the section.
elt(type: string, from: number, to: number, children⁠?: readonly Element[]) → Element elt(tree: Tree, at: number) → Element: Create an Element for a syntax node.

interface InlineParser

Inline parsers are called for every character of parts of the document that are parsed as inline content.

name: string: This parser's name, which can be used by other parsers to indicate a relative precedence.
parse(cx: InlineContext, next: number, pos: number) → number: The parse function. Gets the next character and its position as arguments. Should return -1 if it doesn't handle the character, or add some element or delimiter and return the end position of the content it parsed if it can.
before⁠?: string: When given, this parser will be installed directly before the parser with the given name. The default configuration defines inline parsers with names Escape, Entity, InlineCode, HTMLTag, Emphasis, HardBreak, Link, and Image. When no before or after property is given, the parser is added to the end of the list.
after⁠?: string: When given, the parser will be installed directly after the parser with the given name.

interface DelimiterType

Delimiters are used during inline parsing to store the positions of things that might be delimiters, if another matching delimiter is found. They are identified by objects with these properties.

resolve⁠?: string

If this is given, the delimiter should be matched automatically when a piece of inline content is finished. Such delimiters will be matched with delimiters of the same type according to their open and close properties. When a match is found, the content between the delimiters is wrapped in a node whose name is given by the value of this property.

When this isn't given, you need to match the delimiter eagerly using the findOpeningDelimiter and takeContent methods.

mark⁠?: string

If the delimiter itself should, when matched, create a syntax node, set this to the name of the syntax node.

class Element

Elements are used to compose syntax nodes during parsing.

type: number: The node's id.
from: number: The start of the node, as an offset from the start of the document.
to: number: The end of the node.

Syntax	Description
Header	Title
Paragraph	Text

	/// Add highlighting tag information for this node. The value of
	/// this property may either by a tag or array of tags to assign
	/// directly to this node, or an object in the style of
	/// [`styleTags`](https://lezer.codemirror.net/docs/ref/#highlight.styleTags)'s
	/// argument to assign more complicated rules.
	style?: Tag \| readonly Tag[] \| {[selector: string]: Tag \| readonly Tag[]},

lezer-parser / markdown Goto Github PK

markdown's Introduction

lezer-markdown

Interface

class MarkdownParser extends Parser

interface MarkdownConfig

GitHub Flavored Markdown

Other extensions

Extension

interface NodeSpec

class BlockContext implements PartialParse

interface BlockParser

interface LeafBlockParser

class Line

class LeafBlock

class InlineContext

interface InlineParser

interface DelimiterType

class Element

markdown's People

Contributors

Stargazers

Watchers

Forkers

markdown's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs

`class` MarkdownParser `extends Parser`

`interface` MarkdownConfig

`interface` NodeSpec

`class` BlockContext `implements PartialParse`

`interface` BlockParser

`interface` LeafBlockParser

`class` Line

`class` LeafBlock

`class` InlineContext

`interface` InlineParser

`interface` DelimiterType

`class` Element