char0n / http-request-in-editor

Reference implementation of HTTP Request in Editor Specification https://github.com/JetBrains/http-request-in-editor-spec

HTTP Request in Editor Implementation

What is this?

JetBrains company has come up with with very interesting concept called HTTP Request in Editor Spec. They implemented this spec into their IDEs. You are able to create files with .http extensions with nice syntax highlighting and run HTTP requests directly in your editor. Eventually you can create collection of these .http files, check them in as part of your codebase and share them with the rest of your team. This is really great...but...

There is currently no CLI runner for .http files. From the moment we checked our .http files inside our project we should test if those file reflect reality. This project provides tool for running .http files on your CI.

Warning: Currently we have only implemented the parser which creates documented CST. We're working hard on implementation of runner compatible with JetBrains one. Stay tuned for the runner!

---

This repo contains reference implementation of HTTP Request in Editor Spec parser.

The HTTP Request in Editor Spec is using context-free grammar to present set of production rules. We're using nearley to declaratively map this grammar and generate a JavaScript parser from it.

Parser can parse following syntax and creates CST that can be consumed and executed by various runtime environments. I'll provide a reference implementation of a reference runtime implementation as soon as the parser is finished.

### request 1
POST https://httpbin.org/post HTTP/1.1
Authorization: token

request body 1

### request 2
POST https://httpbin.org/post HTTP/2.0
Authorization: token2

{
  "test": 3,
  "a": "b"
}

### request 3
POST https://httpbin.org/post HTTP/3.0
Authorization: token3

{}

###

Installation

 $ npm i http-request-in-editor

Parser

const { parse } = require('http-request-in-editor');


const http = `
POST https://httpbin.org/post

###
`;

const cst = parse(http);
// now process the CST with your favorite http library

CST

Parser is producing JSON serializable CST. Following HTTP Request in Editor fragment

### post request
POST http://www.example.com HTTP/2.0
# comment
Authorization: token

message body

> {% script %}
<> ./file.json

###

will produce following CST:

(requests-file
  (request
    (requests-line
      (method)
      (request-target
        (absolute-form
          (scheme)
          (literal)
          (hier-part
            (authority
              (host
                (ipv4-or-reg-name))))))
      (http-version))
    (headers
      (header-field
        (field-name)
        (field-value)))
    (message-body
      (messages
        (message-line)
        (message-line)))
    (response-handler
      (handler-script))
    (response-ref
      (file-path))))

CST should be self-explanatory. For more information checkout our CST Types.

Runner

const fs = require('fs');
const { runSpec } = require('http-request-in-editor');

const spec = fs.readFileSync('./spec-file.http');
const result = await runSpec(spec); // returns list of response objects

Development

Fork the repo, clone in and install by

 $ npm i

Edit src/parser/grammer.ne and create a grammar that maps to HTTP Request in Editor Spec.

Compiles src/parser/grammar.ne file into src/parser/grammar.js.

 $ npm run compile

Test parser grammar against predefined fixtures.

 $ npm test

Generate railroad diagrams from src/parser/grammar.ne file.

 $ npm run railroad

Generate random strings that satisfy the grammar defined insrc/parser/grammar.ne file.

 $ npm run unparse

Regenerates Corpus file. Replaces CST S-expression representation with the new one.

 $ npm run corpus:regenerate

Lint the source code.

 $ npm run lint

Notes

1. Multipart-form-data will not be part of resulting CST. It's just a special shape of message-body and requires no special handling even during runtime.

http-request-in-editor's Issues

requestTarget fixtures

No requestTarget fixtures

Create GitHub Action

Fix requestTarget signature

Reorganize fixtures into logical groups

Add support for optional whitespace in HEADER_FIELD

message-line separator

Separator should be translated back when joining message-lines. Don't use \n explicitly.

Utilize prettier

Lint staged

commit lint and husky along with it

Tag support

Improper CST generation

Steps to reproduce:

POST http://[::1]/ - node without a type is generated

Env variables

The variable name may only contain letters, digits, the underscore symbols _, or the hyphen symbols -. Can optionally start with $ character.

Release the v0.1.0

v0.1.0 will contain only parser implementation. Having a parser will allow to implement vendor runtimes.

2 requests-file ?

Describe the bug
when using the end slash in the url, I get 2 "requests-file"

To Reproduce
Steps to reproduce the behavior:

import { parse } from 'https://cdn.skypack.dev/http-request-in-editor';

const http = `
GET http://example.com/

###
`;
const cst = parse(http);
console.log(cst);

[
  {
    type: "requests-file",
    location: 0,
    children: [ { type: "request", location: 1, children: [Array] } ]
  },
  {
    type: "requests-file",
    location: 0,
    children: [ { type: "request", location: 1, children: [Array] } ]
  }
]

request target bug

Describe the bug
the children of request-target node is an array of arrays. is it the expect beheavior?

{
  type: "request-target",
  location: 5,
  children: [ [ { type: "absolute-form", location: 5, children: [Array] } ] ]
}

To Reproduce
Steps to reproduce the behavior:

import { parse } from 'https://cdn.skypack.dev/http-request-in-editor';

const http = `
GET http://example.com

###
`;
const cst = parse(http);

Expected behavior
I expect:

{
  type: "request-target",
  location: 5,
  children: [ { type: "absolute-form", location: 5, children: [Array] } ] 
}

Screenshots

Proper AST nodes/types

Is your feature request related to a problem? Please describe.

Currently our AST is not that well traversable.

Describe the solution you'd like

Wrap every AST node into it's own type so that the AST will be traversable like in babel

Describe alternatives you've considered

Leaving the Status Quo

Additional context

https://github.com/jamiebuilds/babel-handbook/blob/master/translations/en/plugin-handbook.md
https://github.com/graphql/graphql-js/blob/6953f9714b785bbcd89f0103fd73401c06d5f6fe/src/language/ast.js

PR/Issue templates

Dependabot deps update

No method fixtures

Command Line Interface Request

Is your feature request related to a problem? Please describe.
I like this tool a lot. We need to be able to run it as cli. I am unable to find any documentation for running it on terminal. I see a lot of potential for that.
Describe the solution you'd like
It would be nice if we can have the tool be able to be installed as global CLI so that we could run it on terminals. For example, http-request http-request-file.http. This will make the tool more widely adopted.

Describe alternatives you've considered
At present other than using VS Code or Jet Brain's IDEs there is no CLI options.

Additional context
See this description for what I am actually looking for

Properly attribute the visit function in NOTICE file

Documents

Contributing, TOS, conduct, editorconfig

Node 10 EOL

Create parser API

Github actions as CI

Request mention on nearley website in projects section

https://nearley.js.org/

Change license to Apache 2

Create Github Action for marketplace

No fixtures for indented header field value

Sponsors file

Ast -> Cst in corpus

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

Rename repo to http-request-in-editor

Detect ambiguities in representation visitor by throwing error

Utilize eslint

Utilize dependabot v2

Input without empty lines is not parsing.

Describe the bug
Input without empty lines at the end is not parsing.
Simple input like GET https://test.com returns an empty array

To Reproduce

const { parse } = require('http-request-in-editor')


// Working
let input = `GET https://ifconfig.co/json` + '\n\n'

// NOT Working
let input = `GET https://ifconfig.co/json`

const cst = parse(input);
console.log(JSON.stringify(cst))

Expected behavior
CST is returned for input without empty lines at the end of the input

Document AST

Header vs Messages ambiguity

###
POST https://httpbin.org
  /post
  ?search1=value1
  &search2=value2
  #fragment1
  fragment2
Content-type: application/json

{"prop": "value"}

###

Badges to README

Add support for env variables in parser

When composing an HTTP request, you can parametrize its elements by using variables. A variable can hold the values for the request's host, port, and path, query parameter or value, header value, or arbitrary values used inside the request body or in an external file.

Ref: https://www.jetbrains.com/help/idea/exploring-http-syntax.html#using_request_vars