[CONTRIB] Make it easier to work on libdefs about flow-typed

[ryyppy]

Just to sum up this whole discussion:

We want to be able to test specific libdefs with specific tests more easily and independent from existing projects.

We need a command similar to this:

$ flow-typed run-tests --libdef=./redux-saga_1.0.x.js --testFile=./test_redux-saga_1.0.x.js --flowVersion=v0.27.0 --watch

Whenever I change my libdef / test-file, it should re-run the tests (similar to how mocha etc. would do it)
Got this on my radar.

[ryyppy]

@ctrlplusb I agree, IMO this is something which is absolutely required for a 1.0 release... AFAIK @thejameskyle is working on a flow handbook (https://twitter.com/thejameskyle/status/751889868288978944) ... maybe this would be a good opportunity to piggy-back?

[jeffmo]

This is really good feedback. Here's a proposal -- curious to hear what you think:

Currently flow-typed run-tests scans the PWD for a definitions repo and tries to run tests from it based on the file-path naming conventions. What if we extended it to take 3 flags like: --libdef=PATH_TO_LIBDEF, --testFile=PATH_TO_TEST_FILE, and --flowVersion=FLOW_VERSION that let you override the libdef/testfiles/flowversions found by scanning the repo directory?

So for example, say you have some pre-existing libdef over in your project. You could tweak the libdef there, write a test file anywhere you like, and just do:

$ flow-typed run-tests --libdef=./myLibDef.js --testFile=~/pendingTestFile.js --flowVersion=v0.27.0

[ryyppy]

Hey! I need to reiterate this... so e.g. I created some directory with a preconfigured .eslintrc / .flowconfig and I am currently PWDed in there, I would run...

$ flow-typed run-tests --libdef=./redux-saga_1.0.x.js --testFile=./test_redux-saga_1.0.x.js --flowVersion=v0.27.0

So flow-typed would only potentially see redux-saga_1.0.x.js as a lib file and check test_redux-saga_1.0.x.js for typing errors? If so, yes, I would be fine with that....

The only thing, which makes me OCDing, is if there are 100 different styles for all the definitions... so it would still be cool to propose some project-subjective linting to prevent a too diverse code-style in both, the definitions and the commandine tool... that's why I initially thought about a /playground directory or a root-based config.

[ryyppy]

Anyways, I like the CLI command idea, so if we are going to do this, I would be willing to work on this

[jeffmo]

Create a pre-configured playground directory inside flow-typed, which is set up with its own package.json, similar to the cli directory

I'm not sure I understand what this directory would contain. Maybe you can expand on that idea a bit?

if there are 100 different styles for all the definitions...

I'm not a big style-monger so I'll stay mostly out of that discussion (and let you run with it if you feel it's important). That said, I think if we want to enforce linting on libdefs it should be done outside of the run-tests command (i.e. in Travis/npm run lint). That way people can use run-tests on their project-specific libdefs with their project-specific styles until they have it all working -- then they only have to worry about fixing up the final result when sending the pull request.

If you'd like to work on a libdef style-guide, that might be good to discuss this in another issue. I'm not opposed -- I'll likely only be interested in making sure we stay as non-oppressive of all the subjective/non-semantic stuff as possible :) Semantic lints are great, though! ("This variable not defined", etc)

[ryyppy]

Of course :-)
When I try to work in the definitions directory directly and work with the flow-typed run-tests tool, the feedback loop is quite bad, because the test-run has to be done manually all the time.

I would like to never leave my e.g VIM when I iterate on my declarations file.
Neither do I want to fill my error-window with flow-type errors, which are caused by other defintions (since there are maybe files in there which are not compatible with my IDE-version of flow).

Also when I switch to an existing project of mine (which has everything configured and installed like eslint, flow-bin, other packages), it is also hard to work there, because upgrading my flow-version will sometimes cause the flow-errors to appear for completely different files, which are not related to my test* / libdefs itself.

So with a /playground, I would put in a package.json which kinda looks like this:

{
  "devDependencies": {        
    "babel": "^6.5.2",
    "babel-cli": "^6.9.0",
    "babel-core": "^6.9.0",
    "babel-eslint": "^6.0.4",
    "babel-preset-es2015": "^6.9.0",
    "eslint": "^2.7.0",
    "eslint-config-airbnb": "^6.2.0",
    "eslint-plugin-flow-vars": "^0.3.0",
    "eslint-plugin-react": "^4.3.0"
  }
}

And some basic .flowconfig, so whenever I try to develop a libdef, I just check out the project, do an cd playground && npm install and start developing right away inside this directory with any global flow version I prefer...

Maybe we could also just treat that as an example directory, to give contributors some major directions of how a contribution setup may look like (with naming convention etc.)...

The advantage is, that they don't have to think about any babel / eslint setup and can develop right away... with the complimentary run-test parameters, they are able to test the files an as soon as they are happy, they copy it to the definitions directory and create a PR...

Maybe it's just easier when I create a PR and you will see what I mean :-)

Hope it's clear what I am aiming for

[jeffmo]

So as far as I can tell, there are 2 workflows you're pointing out that we're not addressing very well right now:

1. The case where you already have a libdef written in an existing project, and you'd like to contribute it up to flow-typed
2. The case where you would like to write a new libdef to be contributed to flow-typed. Sometimes (usually?) you also want to include the new definitions with an existing project as a way of testing them further.

In the case of (1), the libdef is already sitting in your project somewhere and you basically just want to write some tests that can be contributed with it (and run them), right? I'm not sure I see what a playground would buy you in that case, but please clarify if I'm missing something here.

For (2), I guess I don't see the benefit of the playground directory over just making a new dir under /definitions/npm/ and creating the new libdef there. You don't really need babel/eslint to write a libdef + libdef tests, right?

The thing that you touched on that resonates with me is the fact that you'd like to stay in your editor and not get errors/wait on tests for other libdefs. For that, I think you can already do this with the CLI today: For example, if you're working on a libdef for "my-libdef" and you run flow-typed run-tests my-libdef from within your repo checkout, it should run only the tests for libdefs matching "my-libdef". This should work within vim as well, right?

Sorry for the confusion here, just hoping to break this down to understand.

[ryyppy]

For that, I think you can already do this with the CLI today: For example, if you're working on a libdef for "my-libdef" and you run flow-typed run-tests my-libdef from within your repo checkout, it should run only the tests for libdefs matching "my-libdef". This should work within vim as well, right?

I guess this will be the only proper solution... especially with an --watch option

[jeffmo]

Ooo, yea a --watch option is a good idea

[ctrlplusb]

On this same topic it would be great for the most experienced of the definition contributors to establish a brief set of guidelines/standard/pitfalls for creating definitions. I've seen a various mix mash of styles when looking through the existing type definitions.

There is a lot to consider:

• global vs local types
• modules
• considering all the various export locations of a library
• avoiding name conflicts
• dependencies
• versioning strategies
• effective testing
• api coverage

A step by step guide would be great. "A beginners guide to type definitions". I honestly feel if we can provide as much education on this process as possible we will see an increase in community contributors. Myself included.

I am happy to contribute to this guide, but I consider myself very much an apprentice at this stage. I have added flow to 3 projects I am developing in parallel however, so hopefully I can skill up fast.

[ctrlplusb]

That is brilliant news! James is awesome at handbooks. His babel handbook got me up and running with ASTs in no time.

[jeffmo]

On this same topic it would be great for the most experienced of the definition contributors to establish a brief set of guidelines/standard/pitfalls for creating definitions.

It's not exactly brief, but I started jotting down some of the things I try to look for when accepting PRs here: https://github.com/flowtype/flow-typed/blob/master/CONTRIBUTING.md#library-definition-best-practices

Right now I think it only talks about using any vs mixed, a bit about namespacing globals, and a bit about the minimum requirements for tests -- but I think you've listed out a pretty good set of additional topics that we should add. Also, having a better formatted step-by-step guide would be really cool, too (cc @thejameskyle who's been thinking about this a bit).

[gantoine]

Closing as CONTRIBUTING.md and the docs have evolved considerably since this issue was first opened.