shicks / proposal-multi-backtick-templates

TC39 Proposal for multi-backtick templates with automatic margin stripping

Multi backtick templates with automatic margin stripping

Champions: @jridgewell, @hemanth

Author: @mmkal

Status: draft

Original TC39 thread

Problem

Javascript template literals force users to choose between strange-looking code, have the resultant outputs include unwanted indentation, or rely on third-party dependencies to "fix" template margins. They also require escaping backticks, limiting the ability to put code samples inside templates for languages which include backticks (e.g. javascript/markdown/bash).

The current options:

Strange-looking code

class MyClass {
  print() {
    console.log(`create table student(
  id int primary key
  name text
)`)
  }
}

Sensible-looking code, with unwanted indents in output

class MyClass {
  print() {
    console.log(`
      create table student(
        id int primary key
        name text
      )
    `)
  }
}

This prints:

      create table student(
        id int primary key
        name text
      )
      

Note the newlines at the start and end of the output, and a margin on each line.

With a library

A commonly used one is dedent:

import dedent from 'dedent'

class MyClass {
  print() {
    console.log(dedent`
      create table student(
        id int primary key
        name text
      )
    `)
  }
}

Now the input and output look about right, but we've had to add a runtime dependency. This makes what could be pure data less portable. You can no longer copy-paste snippets, you have to copy-paste snippets, install an npm package, maybe install the types too, import it and then use it. This solution also cannot be used in ecmascript-compatible data formats like json5.

Similarly, jest-inline-snapshots automatically "dedent" literal snapshots at runtime. There is also a babel plugin which moves the "dedenting" to compile-time.

This presents a problem when template expression tags also need to be used:

const raw = dedent`
  query foo {
    bar
  }
`;

gql`${raw}`; // <- this doesn't work right.

To get around this, we have to rely on the tagged template being callable as a regular function:

gql(raw)

This doesn't allow for customized behaviour of expressions. For example, slonik protects against sql injection by automatically sanitizing expression parameters. To solve that, we could extended dedent to compose with other tags:

const query = dedent(sql)`
  select *
  from students
  where name = ${name}
`

But there are negative implications to this. Quote from the original tc39 thread:

[This approach] is not free (some runtime processing and WeakMap lookup), and will conflict with the "templateness" in proposals like https://github.com/tc39/proposal-array-is-template-object.

It'd be better this could be supported at the language level, to avoid:

• all of the above complexity
• the runtime implications of squashing so much parsing responsibility into dedent or similar libraries
• inconsistencies between implementations of "dedenters" (jest vs dedent vs babel-plugin-dedent)
• the need for coupling dedent to the implementation particulars of other tagged template literal functions
• the need to teach formatters like prettier that they can safely adjust the margins of some templates, but not others (and even in these cases, the formatter would have knowledge about the context and implementation details of the various dedenting libraries):

  • dedent`...`

  • dedent(anything)`...`

  • .toMatchInlineSnapshot(`...`)

There are several other libraries which each have a very similar purpose (and each behave slightly differently). Some examples:

• https://www.npmjs.com/package/strip-indent
• https://www.npmjs.com/package/trim-margin
• https://www.npmjs.com/package/stripmargin

Proposed solution

Allow specifying triple-, quintuple, or septuple, or any-odd-number-uple backtick-delimited literals, which behave almost the same as a regular backticked template literal, with a few key differences:

• The string is automatically "dedented", along the lines of what the dedent library does. A simple strawman algorithm:
  • the first line (including the opening delimiter) is ignored
  • the last line (including the closing delimiter) is ignored if it contains only whitespace
  • the "margin" is calculated using the whitespace at the beginning of the first line after the opening delimiter
  • that margin is removed from the start of every line
• The opening delimiter must be immediately followed by a newline or the closing delimiter
• The closing delimiter should only contain whitespace between it and the previous newline
• Backticks inside the string don't need to be escaped

The examples above would simplify to something like this:

class MyClass {
  print() {
    console.log(```
      create table student(
        id int primary key,
        name text
      )
    ```)
  }
}

This will output the template with margins stripped:

create table student(
  id int primary key,
  name text
)

Custom expressions would work without any special composition of tag template functions:

const query = sql```
  select *
  from studients
  where name = ${name}
```

We can also avoid the need for escaping backticks when they're needed inside the template:

const printBashCommand = () => {
  console.log(```
    ./some-bash-script.sh `ls`
  ```);
};

Using more backticks allows for triple-backticks inside templates without escaping:

const getMarkdown = () => {
  return `````
    # blah blah

    ```json
    { "foo": "bar" }
    ```

    some _more_ *markdown*
  `````;
};

The behavior when later lines lack the whitespace prefix of the first line, is not yet defined:

const tbd = ```
  The first line starts with two spaces
but a later line doesn't.
```

In other languages

• Java - text blocks using triple-quotes
• Scala - multiline strings using triple-quotes and .stripMargin
• Python - multiline strings using triple-quotes to avoid escaping and textwrap.dedent
• Jsonnet - text blocks with ||| as a delimiter
• Bash - <<- Heredocs
• Ruby - <<~ Heredocs
• Swift - multiline string literals using triple-quotes - strips margin based on whitespace before closing delimiter

Implementations

A very basic POC polyfill for babel is implemented in this repo, mostly as a strawman to make it easier to discuss the particulars of how indenting should work - so it's as-yet unpublished.

See the tests for proposed functionality - that is, how triple-backticks can be transpiled to current ecmascript. These don't include any parsing changes, so there are no examples of unescaped backticks inside templates.

TODO

• babel parser plugin allowing for unescaped backticks
• a patch to typescript library types, since typescript will get upset about trying to use template literal tag as a template literal tag.

Q&A

Is this backwards compatible?

This could be partially implemented with no syntax changes, since it's technically already valid syntax:

```abc```

Is equivalent to

((``)`abc`)``

Where the empty-strings are being used as tagged template functions. i.e. when run, this code will try to use the empty string

(``)

as an template tag, passing in 'abc', the return value of which is then used as another es string tag which receives the empty string. Obviously, none of that will currently work at runtime, because an empty string is not a function. So no functioning code should be affected by this change.

Some parsing changes would be needed to allow for unescaped backticks inside triple-backticked templates.

Why not use a library?

To summarise the problem section above:

• avoid a dependency for the desired behaviour of the vast majority of multiline strings (dedent has millions of downloads per week)
• make code snippets more portable
• improved performance
• better discoverability - the feature can be documented publicly, and used in code samples which wouldn't otherwise rely on a package like dedent, which is on major version 0 without an update in three years
• establish a standard that can be adopted by JSON-superset implementations like json5
• give code generators a way to output readable code with correct indentation properties (e.g. jest inline snapshots)
• support "dedenting" tagged template literal functions with customized expression parameter behaviour (e.g. slonik)
• allow formatters/linters to safely enforce code style without needing to be coupled to the runtime behaviour of multiple libraries in combination