Type annotations in comments? about flow

[bcherny]

@Raynos @jareware @andrewrota

what's the benefit of using a new, custom syntax as opposed to already widely adopted jsdoc?

/**
 * Foo
 * @param {String} foo
 * @param {Number} bar
 * @return {Array} An array
 */
function foo (foo, bar) {}

Or (not sure if flow is this smart):

/**
 * Bar
 * @param  {Array} foo An array
 * @param  {Array<Array>} bar An array of arrays
 * @param  {DOMElement} baz An element
 * @return {Promise}
 */
function bar (foo, bar, baz) {}

[kegsay]

flow-jsdoc now supports inline syntax of the form:

//: (string): Object
function foo(bar) {
  return {};
}

Parameters are paired in order and will currently panic if there are mismatched lengths. I'll probably add a flag to make that optional (and it just won't add types to subsequent params / ignore additional types in comment).

[cletusw]

It just makes so much more sense to me to add these type annotations in comments. Then you can reuse existing comment-stripping build tools instead of inventing a new one. The only benefit I can see of using a new syntax is that it forces the compile step (and its accompanying type check).

[kegsay]

For people who want a 1-line comment of the form:

/*: (x: string, y: number): boolean */
function(x, y) {
 // ...
}

Presumably this is desirable due to compactness. If so, presumably the following would be even better (else you need to make sure if you rename a variable that you also rename the comment form):

/*: (string, number): boolean */
function(x, y) {
 // ...
}

If there is enough interest in a one-line form, I should be able to very easily fold this behaviour into flow-jsdoc given I'm already parsing comments at this stage. This would effectively map:

/*: (string, number): boolean */
function(x, y) {
 // ...
}

Into:

/*: (string, number): boolean */
function(x: string, y: number) : boolean {
 // ...
}

[jareware]

With regard to the discussion above, while flotate doesn't (yet) support full-blown JSDoc, there's now also an alternative syntax for functions/methods, so that in addition to:

function foo(x /*: string */, y /*: number */) /*: boolean */ {
    return x.length * y === 5;
}

the following is also supported:

/*: (x: string, y: number): boolean */
function foo(x, y) {
    return x.length * y === 5;
}

which is nice in keeping the annotations & code separate.

[ronkorving]

I for one would love to see support for the flotate syntax that @ngduc refers to there. It's the one thing I hoped I could use (the inline comments are a bit messy imho), and would be a fair replacement for JSDoc (for some of my use cases), and much more compact.

[samreid]

The preceding one-line syntax examples are misleading because they lack documentation. For instance, a documented version of the preceding example might be something like:

  /**
   * Makes a prediction using decision trees whether a person would have survived a trip on the Titanic.
   * @param {string} x - last name of a person who rode on the Titanic (case insensitive)
   * @param {number} y - age of the person who rode on the Titanic in years
   * @return {boolean} - true if the prediction from the model is survival
   */
  //: (string, number): boolean
  function(x, y){
    // ...
  }

The one-line syntax (or even flow annotations) is redundant, let's comment our code and add JSDoc type checking for Flow.

[metrofun]

+1
JSDoc would be the right direction for increased interoperability of Flow. Having one more magic syntax for types is quite obtrusive. Right now JSDoc is a clear advantage of Closure Compiler when it comes to type checking.

[ngduc]

Does flow support flotate one line syntax yet? I haven't found an example online:

/* @flow */
/*: (x: string, y: number): boolean */
function foo(x, y) {}

[Pyrolistical]

@kegsay I love it! We can even make it more compact by introducing a single line comment syntax:

//: (string, number): boolean
function(x, y) {
 // ...
}

[kamek-pf]

JSDoc would be nice indeed.
What's the status on this ? Is it going to happen at all ?

[quantuminformation]

Any update on jsdoc being wired to flow?

[arthurschreiber]

This is not a documentation issue. flow port can convert your code that uses "docblock" based annotation into the flow annotation syntax:

/* @flow */

/**
  @param {string} x
 */
function foo(x) {
  return x*10;
}

foo("Hello, world!");

will give the following output:

$ ./flow port examples/01_HelloWorld/hello.js 
/Users/arthur/Downloads/flow/examples/01_HelloWorld/hello.js
--- old
+++ new
@@ -3,7 +3,7 @@
 /**
   @param {string} x
  */
-function foo(x) {
+function foo(x: string) {
   return x*10;
 }

But these annotations can't be directly used for type checking. But in theory, it should be possible to change flow to do this conversion in-place.

[dchambers]

Avoiding having to use a transpiler is a big deal in making this usable on existing projects, and while I think @cletusw's suggestion was also really neat, for example:

function foo(a /*: string*/, b /*: number*/) /*: number*/ {
  // function body...
}

in terms of working with existing idiomatic JavaScript, which is one of Flow's stated goals, then supporting jsdoc annotations would be a huge win also, so:

/**
 * @param {String} a
 * @param {Number} b
 * @returns {Number}
 */
function foo(a, b) {
  // function body...
}

[bcherny]

@JaRail

While it's always easy to make feature requests, I'd personally prefer a mix & match approach, where Flow can consume both styles of type info.

I agree completely. Flow has a parser and a validator - the validator should be pluggable, so we can support multiple existing formats:

• Flow
• Jsdoc
• Closuredoc
• Coffeedoc
• jsig
• Any future dialect
• Typescript, Dart?

The validator should be annotation syntax-agnostic, so long as the output produced by the parser is the same regardless of source language/annotation style.

[dgreensp]

I'd like to echo what @ronkorving said. Being able to annotate a function with a single flow comment would be a huge benefit and make it more likely my team would adopt Flow. Flow comments (as opposed to real annotations) are super useful not just to avoid preprocessing but also to not block on editor support on a diverse team.

The expectation that this would be implemented comes from the fact that, in theory, flotate has been deprecated and its functionality folded into Flow, but this aspect of flotate apparently did not make it over.

[Raynos]

Having restrict rules around comments is fine.

// foo(a: mixed, b: mixed) : void
function foo(a, b) {
  // ...
}

The real problem in parsing comes when you combine the type definition and documentation in one comment block.

Supporting just type definitions in a single comment statement where the author of the type definition is not allowed to leave "comments" in the comments or "documentation" in the comments makes this problem far simpler.

We can also use a flow: prefix as mentioned above and warn about any flow: comments that were not parsed properly.

[concavelenz]

It is actually fairly straightforward for type annotations. You can steal
the logic from the closure compiler parser.
On Nov 18, 2014 4:14 PM, "Gabe Levi" [email protected] wrote:

Comments are kind of a parsing nightmare. Knowing which node a comment
belongs to is a Hard Problem, and often is solved by "guessing" based on
whitespace. The way around this is to stick to conventions, but then you
need something to enforce these conventions or risk silently misbehaving
when they're broken.

So yeah, adding type annotations to the grammar simplifies things, though
it does add a build step. It's a definite tradeoff, though.

—
Reply to this email directly or view it on GitHub
#3 (comment).

[Raynos]

@bcherny

The benefit is that the JSDoc language is different from the flow language.

Using something that looks like JSDoc but is not JSDoc is confusing.

With that said you might as well use a comment language that maps as cleanly onto flow as possible. This means you have zero legacy baggage from JSDoc and zero legacy baggage from JavaDoc.

Coupling yourself to JSDoc has no benefits and just limits flow.

[Raynos]

@bcherny

JSDoc has no way to express overloaded functions. i.e.

((x: number) => void) & ((x: string) => void)

Trying to express this syntax in the existing "framework" of a comment block with a list of new line seperate @param statements is going to be frustrating.

What you really want is a one line comment above the function that is the exact same as the flow language itself.

[gabelevi]

Ok, you guys convinced us! We've merged @jareware's awesome flotate syntax upstream, directly into Flow! Here's a blog post about it and we'll write proper documentation after we've caught up on sleep :)

[kegsay]

@heyarne :

There's support for converting existing code using jsdoc syntax into the flow syntax, which could then be used for checking, is that correct?

As far as I know, the flow binary does not support this. However, I have a project which will do this conversion for you: Kegsay/flow-jsdoc.

Does that conversion handle modules?

No, it doesn't add in modules which are required for the type checking to work. It's an addition I want to add in the future though. The tool can operate on an entire directory, but each file is currently treated in isolation.

And how does it handle mixed code where some files are written using the flow syntax and some use jsdoc?

Badly. Esprima doesn't support Flowtype syntax last time I checked, so the converter won't think it is valid JS.

[kegsay]

@samreid That is exactly the rationale behind flow-jsdoc. The one line syntax is intended as an alternative, I have no desire to replace the exisiting logic to parse JSDoc. One line syntax works particularly well for private methods which people may not be documenting in JSDoc.

@Pyrolistical I would probably support both forms, as I find it somewhat magical if a tool treats block/line comments differently to each other when the actual language doesn't.

As for "Flow may one day sprout X", I'll cross that bridge if it ever happens. Worst case is a major version bump and a disclaimer that it only works on Flow vX and above. I'm not going to do nothing just because of speculative features.

[cletusw]

Hmm, what if you ignored jsdoc comments and only respected inline comments, like here? That shouldn't be too hard to parse, right? You could potentially even give a warning for any comment inlined that way that didn't parse correctly to prevent silent misbehavior. Or if that inline comment syntax is already used in lots of places maybe require a "flow:" prefix, e.g. function foo(/*flow:string*/ text, /*flow:number*/ count) /*flow:string*/ {}

[cletusw]

In other words, why is

function foo(a: mixed, b: number): void { ... }
var x: boolean;
class Bar {
  y: string;
}

parseable, but not

function foo(a /*: mixed*/, b /*: number*/) /*: void*/ { ... }
var x /*: boolean*/;
class Bar {
  y /*: string*/;
}

?

(Not that I recommend that exact syntax. Just wondering :-)

[jareware]

For @MisterSpeaker et al, I put together https://github.com/jareware/flotate for just this purpose. That is, using Flow with standard JavaScript syntax.

[ngduc]

flotate looks great! Please support JSDoc syntax also. Thanks.

[Pyrolistical]

I agree with @dgreensp. If support was added for a single line comment above the functions I would propose my team to use flow

[kegsay]

@kegsay's tool looks nice, but doesn't support JSX

Does now!

[Raynos]

👍 Reading https://github.com/facebook/flow/blob/master/src/typing/comments_js.ml it looks like its supported.

[benjamingr]

+1 for this. Would definitely be helpful in evaluating migration.

[mhart]

@Raynos Oh cool – just didn't see it mentioned in the docs (maybe it's not yet?)

[mhart]

Oooh, think I found some tests: https://github.com/facebook/flow/blob/master/tests/docblock/docblock.js

Might change this to be a documentation issue (edit: done)

[Raynos]

cool :) Sounds like we just need documentation for this.

[mhart]

Ah cool, makes sense.

Ok – will keep this issue open then

[Raynos]

Ah I see. Fair enough.

[avikchaudhuri]

Hah, at Facebook we have existing code using a comment syntax that we try to convert into Flow annotations. This feature should be treated as experimental, not sure if our comments syntax is anything close to standard (jsdoc?)

[gabelevi]

Comments are kind of a parsing nightmare. Knowing which node a comment belongs to is a Hard Problem, and often is solved by "guessing" based on whitespace. The way around this is to stick to conventions, but then you need something to enforce these conventions or risk silently misbehaving when they're broken.

So yeah, adding type annotations to the grammar simplifies things, though it does add a build step. It's a definite tradeoff, though.

[samreid]

I agree it would be very useful for flow to support @param annotations (as an alternative to the provided :type syntax) in order to deduce the types. This will make it significantly easier for existing projects to benefit from Flow, and will avoid the transpiler step for live code + the compilation step for minified code.

[j201]

Using comment type annotations might be necessary if using simple compile steps like browserify or sweet.js, where JS type annotations are relevant but adding them outside of comments would break the compile steps.

[jareware]

@cletusw's suggestion, and the rest of the arguments on this issue are extremely sound. For a bit more typing (and maybe a tad uglier code), you could get all the benefits of Flow without having to leave standard JavaScript syntax compatibility. This is a HUGE win in terms of bringing existing projects onboard. Also, many people may not want to make the commitment even in greenfield projects, as going over to Flow (or TypeScript) will need support from most other tools and IDE's you might want to use, and will massively limit the ecosystem available to you. Finally, as stated above, this aligns perfectly with Flow's stated goal of being able to work with any JS, and helping out with type annotations only gradually and when needed.

As to the concrete proposal, I would go even further and drop the colons:

/* @flow */
function foo(a /*mixed*/, b /*number*/) /*void*/ { ... }

...as it's highly unlikely inline comments would exist at those exact locations unless they're meant for this exact purpose.

If @avikchaudhuri or @gabelevi could comment as to how likely adding a 100% JS compatible syntax to the project would be, that'd be hugely appreciated! Because if not, someone in the community can take up building a pre-processor for instance.

[benjamingr]

+1 for @Raynos 's suggestion. I think it's very clear and allows smoother migration.

[andrewrota]

+1, I agree that this would be an awesome feature with whichever comment syntax was most feasible.

Just to add another possible syntax, the inline Closure Compiler annotations look pretty similar to flow except for the return type before the function rather than after. I don't know if the logic from Closure Compiler to parse this would be helpful at all in porting the capability to Flow, though.

function /** string */ foo(/** string */ a, /** number */ b) {}

Thanks for considering this feature!

[darrenderidder]

+1, I made the same comment on HN when the announcement came out. Annotations belong in annotations (aka. comments); transpiling is quixotic.

[andrewrota]

@bcherny, I think JSDoc syntax would be great, I just thought it would be mentioning the more inline syntax (especially since Flow's type annotations are inline). Closure Compiler supports both the JSDoc style and the inline comment style.

[bcherny]

@Raynos I already use jsdoc for docs and automated interface testing, so my ideal workflow would be to continue using jsdoc, and add in flow as a static typechecking tool that makes use of my existing jsdoc annotations.

The parser is already decoupled from the rest of flow's source, and it would be cool to have a few different parsers available (flow, jsdoc, closure, etc.).

Coupling yourself to JSDoc has no benefits and just limits flow.

Can you explain this a bit more?

[bcherny]

@Raynos By "overloaded" you mean a function which can return multiple types, with params that might accept multiple types? Jsdoc does support that use case, but you're right that it does not support mapping inputs to outputs.

/**
 * @param {String|Number} foo
 * @param {Any} bar
 * @return {Array|Object} An array or an object
 */
function (foo, bar) {}

[Raynos]

@bcherny

overloaded means ((x: number) => number) & ((x: string) => string). This is different from having parameters that are union types and having return values that are union types.

You are correct that the main difference is, with overloaded functions you can infer the correct return type based on the input types.

This is one of many advance features flow supports that are hard to express in jsdoc.

A flow comment system should have a flow based language.

Having external support for flow converting JSDoc into a subset of the flow type system seems reasonable.

While JSDoc certainly has some holes, I'm not sure why pulling in additional type info from JSDoc would be a problem. I think it's also reasonable to expect JSDocs to continue to improve over time. For reference, here's the Google Closure issue to define/add support for intersection types.

google/closure-compiler#202

@Raynos What do you mean by external support? I'm not sure if you're suggesting

• the Flow runtime supports importing/referencing JSDoc types from other modules, eg consuming jQuery externs, or
• a JavaScript+JSDoc -> Flow transpiler is written, completely independent of the Flow runtime

While it's always easy to make feature requests, I'd personally prefer a mix & match approach, where Flow can consume both styles of type info. If both JSDoc and Flow type annotations are provided, Flow could

• ignore the JSDoc,
• error if the type information provided by the Flow annotation is not a subset of that provided by the JSDoc annotation
• enforce the intersection of two

So with bcherny/Raynos's example, any of the above three options would result in enforcement of the overloaded num->num and string->string behavior.

As an alternative to writing a transpiler, perhaps Flow could provide an addon hook? That would allow a 3rd party to plug in their own parsing/typing components. There could be an addons for generic JSDoc, for closure-style, etc. That would keep it very firmly distinct from the core Flow language while also giving the community the flexibility to play around..

[natew]

Just out of curiosity, could flow potenitally then be used as a jsdoc/documentation generator? I'd find that somewhat useful for getting started on writing docs for a project that's already got a bit of code.

[CrabDude]

+1

This would allow compatibility with ES6 transpiled projects (e.g., traceur). Flowtype isn't yet a drop-in replacement for an ES6 transpiler since it's ES6 support is limited.

+1 for me too. Is anyone actually working on this? I don't want an "almost-js -> real js" transpile step in my personal projects, so I'd be keen to follow along/help out.

[mhart]

@jareware Just had a browse of the docs, looks great! I guess it was probably easier to come up with a new syntax rather than try and support JSDoc et al from the get go?

I look forward to trying this out – and hopefully soon we'll be able to check code without the compile/temp-dir step!

[jareware]

@mhart yeah, I gave up on JSDoc pretty quickly after realizing how hard it'd be to match up all params with their types, how much repetition that'd introduce, and that (as @Raynos pointed out above as well) it still wouldn't cover e.g. overloaded function signatures well.

By using the exact same positions as Flow does, and reserving a prefix for the comments (/*:) the conversion is actually dead-simple, as you don't have to care about where the annotations are encountered, you just strip the surrounding /* and */ and you're done. :)

I'm hoping something along these lines eventually ends up in Flow itself, at which point the preprocessor obviously becomes unnecessary!

[Deathspike]

+1 as it would avoid new non-JavaScript syntax and flow through existing annotated libraries. Arguments have been raised that not every case is covered in JSDoc, and while that is true, it would make flow useful for projects without build steps and using regular JS (e.g. almost all node libraries out there).

[yonkeltron]

+1 for this feature. Any of the proposed syntax options would immediately speed my adoption of flow since it is unreasonable for my team to convert existing projects to typed JS right now.

[molszanski]

Flotate one-line-syntax looks like a much more prefereble solution.
It will not break established workflows, testing routines, IDE support, existing code etc.

Also, it looks like it will be easier to integrate flow with things like coffeescript.

[benjamine]

@jareware that's neat! much better readability,
although param names get duplicated, but it's a great option

[yonkeltron]

@jareware Thanks so much for telling us about flotate! I can't wait to use it. As the primary author, do you think that flotate provides evidence that type annotations in comments constitutes a viable method of expressing type information? Seeing as how you went ahead and wrote your own tool, do you think flow proper should support type annotations in comments?

[jareware]

@yonkeltron Definitely viable! I guess the question is more about whether type checking for JavaScript is useful or not. I think it is, and that Flow is currently the best too for the job. flotate just bridges the small syntactic gap to standard JS. For instance, I'm currently gradually adding type information to an existing project using these tools, and have found them helpful!

As to the second question, I think that's something they're considering, and I'd be glad to have the preprocessor become unnecessary. :)

[mhart]

Yessss, excellent news!

[mhart]

Going to close this – if anyone's got issues with the syntax or anything else they can open up separate issues.

[jareware]

This is brilliant! 🙇

[quantuminformation]

Good job!

[jareware]

@ngduc, I don't think it was ever slated for inclusion. And to be fair, while personally I find that one-line syntax very convenient, it's quite different from the standard Flow syntax, and actually isn't as widely applicable as the standard one.

If the team were to consider the addition of completely new syntaxes, I'm guessing JSDoc support would be the first thing to shoot for.

[heyarne]

There's support for converting existing code using jsdoc syntax into the flow syntax, which could then be used for checking, is that correct? Does that conversion handle modules? And how does it handle mixed code where some files are written using the flow syntax and some use jsdoc?

If all that could work seamlessly it would be a killer feature, enabling correct autocompletion and type-checking for existing well-documented libraries, so I could just import foo from 'external/library' and have the documentation right in my editor. 🚀

[neemzy]

Hi,

May I ask whether you guys are considering parsing/supporting JSDoc comments to infer type info from it? Using yet another external tool's kinda cumbersome, it means copying my whole code to a third place (adding to original source and browser-ready code) just for the sake of type-checking it, and it also has its own drawbacks (I can only imagine resolving require calls when the files aren't in their original location anymore is a kinda small but still very real PITA, and that's only the first thing off the top of my head).

@kegsay's tool looks nice, but doesn't support JSX for starters, which is something I need: using it right now would mean setting up a different flavor of my JS build to compile JSX but not remove comments or anything, copy it to a specific location while converting JSDoc to Flow syntax with said tool, and if everything is OK, run my standard build to generate browser-ready code as I already do. I love JS tooling, but this is one step too far for me.

My codebase is full of JSDoc, which indeed is the language's standard type notation. It may not cover everything your own syntax does, but will do the job for a lot of use cases. C'mon, Facebook fellas wrote brand new PHP specs, don't tell me you can't contribute to JSDoc ;) (jk ofc)

I have no idea of the required amount of work this needs, but I'm positive this is the way to go if you want existing, large codebases to take the leap (I want mine to anyway).

[dgreensp]

My only suggestion on the flow-jsdoc front is consider that Flow could one day support flotate one-line syntax. The currently proposed flow-jsdoc feature (which is probably best discussed in that repo, not here) is yet another meaning of /*: and would cause trouble in that case.

[samreid]

@kegsay I agree with everything you said, keep up the good work!