maxim-mazurok / google-api-typings-generator Goto Github PK

Some APIs (storage and vision) produce interfaces such as Object or Symbol which is prohibited by tslint:all rule ban-types, used in dtslint.
I think it's completely normal to have them, since they are not in the global namespace and don't interfere with @types/node or anything like this.
So, we have to find a way to add ignore-next-line comments in a good way.
I've tried to do it quickly, but formatting is off.
Probably this should be done after we will stop producing formatted types and start running prettier on them instead.

We should create a curated list of supported APIs that actually need our typings.
Our main focus should be on APIs that have client (browser) library.

Actually, I was just referring to temp directory path. It might nice to keep both of these temporary things (DT fork and Google API JSON) in the same temp directory for consistency.

Oh, got it. I agree. We might want to have this structure:

tmp
  - api-discovery-json
  - dt-fork

I will create a task to find the best way of getting/creating tmp folder (require('os').tmpdir() or something like this).

Originally posted by @Maxim-Mazurok in #74

For example:

interface Record {
    /**
     * Key defines all of the unique querying parameters needed to look up a user
     * experience record.
     */
    key?: Key;
    /**
     * Metrics is the map of user experience data available for the record defined
     * in the key field. Metrics are keyed on the metric name.
     *
     * Allowed key values: ["first_contentful_paint",
     * "first_input_delay", "largest_contentful_paint",
     * "cumulative_layout_shift"]
     */
    metrics?: Record<string, Metric>;
}

is generated for gapi.client.chromeuxreport.

interface Record overrides built-in Record<K,T> which results in error Type 'Record' is not generic.

We should check that our interfaces do not override any TS specific terms/words. Utility Types are great starting point. Ideally, we should get these types list from TS dynamically instead of hardcoding.

gts v1 uses prettier v1, which doesn't support private class properties.
So, we have to upgrade gts to v2 which uses prettier v2.
Until then, we have to manually keep prettier v2 in package-lock.json
Instead of upgrading to gts v2 we can set up prettier and aslant on own own.

If you have not already, can you require the test and lint status checks for master?

It will look something like this under Settings > Branches:

Please, update PR template, current SHA ab3f84d44f18d1f33cc79c5d857067b939813c7e doesn't match actual SHA ab3f84d44f18d1f33cc79c5d857067b939813c7e

Here's an idea that we might benefit from using modules instead of namespaces.
Let's give it a try and see how it will workout.

DefinitelyTyped/DefinitelyTyped#43293 (comment)

In generated readme we use gapi.client.spreadsheets.batchUpdate instead of gapi.client.sheets.spreadsheets.batchUpdate. And this applies to other types aw well.

Template should be changed.

Build failed: https://github.com/Maxim-Mazurok/google-api-typings-generator/runs/523001083?check_suite_focus=true#step:5:195
Investigate.
Related to #88 (probably this is the cause of #88)

Please, update PR template, current SHA ab3f84d44f18d1f33cc79c5d857067b939813c7e doesn't match actual SHA 5e283ccee07a788a471cfd554b845215de74a549

Hi! Could anyone assist me with the types for google calendar API? I'm trying to use it in my Angular 9 app and couldn't succeed in it. I get an error Namespace 'gapi.client' has no exported member 'calendar' when importing a type like so import Event = gapi.client.calendar.Event;, however, I can use all of the interfaces if I suppress the error with ts-ignore. These are the types I've installed:

• "@types/gapi": "^0.0.39"
• "@types/gapi.auth2": "^0.0.51"
• "@types/gapi.client.calendar": "^3.0.4"

Do we actually need this?

function load(name: "chromeuxreport", version: "v1"): PromiseLike<void>;
function load(name: "chromeuxreport", version: "v1", callback: () => any): void;

for each API type definition?

As per conversation with @orta:

DT merge-bot logic is here: https://github.com/DefinitelyTyped/dt-mergebot

if you arbitrarily change a single test file you can get it easier to auto-merge

You'll need the bot to also say "OK to merge" or something like that after CI (~1 hr) too
Perhaps make an issue on https://github.com/DefinitelyTyped/dt-mergebot/issues about asking for a way to declare that some modules are auto generated

So, we have to:

• add change to tests (probably, revision number) - this actually seems optional
• create second GH account that will open these PRs (to get hasFinalApproval and Merge:Auto)
• respond to bot request to auto-merge (post Ready to merge from @Maxim-Mazurok )

Error occurred during this build: https://github.com/Maxim-Mazurok/google-api-typings-generator/runs/499347449?check_suite_focus=true#step:5:201
Investigate and fix

I'm not sure I see the value in updating types when there is only a revision number change. Thoughts?

It would be much easier if we just check during generation/publication to types branch is we're publishing revision which is never than current revision. So it will never decrease.

Originally posted by @Maxim-Mazurok in #74

After octokit/types.ts#40 is resolved, update @octokit/types in package-lock.json

Related to #126

Build failed on linting SQL typings: https://github.com/Maxim-Mazurok/google-api-typings-generator/runs/505969383?check_suite_focus=true#step:5:974

Further investigation required. I have API changes log on server (#25 script still runs on my server, apparently)

Fix this: isaacs/github#1804
CI log: https://github.com/Maxim-Mazurok/google-api-typings-generator/runs/670491014

What worked: full colne of fork, fetch master without shallow

i.e.

$ npm run dtslint types/gapi.client.adexchangebuyer

> google-api-typings-generator@ dtslint /Users/namoscato/dev/git/google-api-typings-generator
> dtslint "types/gapi.client.adexchangebuyer"

Error: /Users/namoscato/dev/git/google-api-typings-generator/types/gapi.client.adexchangebuyer/gapi.client.adexchangebuyer-tests.ts:35:52
ERROR: 35:52    expect  TypeScript@next compile error: 
Expected 1 arguments, but got 0.
ERROR: 83:55    expect  TypeScript@next compile error: 
Expected 1 arguments, but got 0.
ERROR: 128:60   expect  TypeScript@next compile error: 
Argument expression expected.
ERROR: 129:13   expect  TypeScript@next compile error: 
Argument of type '{ accountId: number; adChoicesDestinationUrl: string; adTechnologyProviders: { detectedProviderIds: string[]; hasUnidentifiedProvider: boolean; }; advertiserId: string[]; advertiserName: string; ... 25 more ...; width: number; }' is not assignable to parameter of type '{ alt?: string | undefined; fields?: string | undefined; key?: string | undefined; oauth_token?: string | undefined; prettyPrint?: boolean | undefined; quotaUser?: string | undefined; userIp?: string | undefined; resource: Creative; }'.
  Object literal may only specify known properties, and 'accountId' does not exist in type '{ alt?: string | undefined; fields?: string | undefined; key?: string | undefined; oauth_token?: string | undefined; prettyPrint?: boolean | undefined; quotaUser?: string | undefined; userIp?: string | undefined; resource: Creative; }'.
ERROR: 1082:60  expect  TypeScript@next compile error: 
Argument expression expected.
ERROR: 1083:13  expect  TypeScript@next compile error: 
Argument of type '{ proposals: { billedBuyer: { accountId: string; }; buyer: { accountId: string; }; buyerContacts: { email: string; name: string; }[]; buyerPrivateData: { referenceId: string; referencePayload: string; }; ... 18 more ...; sellerContacts: { ...; }[]; }[]; webPropertyCode: string; }' is not assignable to parameter of type '{ alt?: string | undefined; fields?: string | undefined; key?: string | undefined; oauth_token?: string | undefined; prettyPrint?: boolean | undefined; quotaUser?: string | undefined; userIp?: string | undefined; resource: CreateOrdersRequest; }'.
  Object literal may only specify known properties, and 'proposals' does not exist in type '{ alt?: string | undefined; fields?: string | undefined; key?: string | undefined; oauth_token?: string | undefined; prettyPrint?: boolean | undefined; quotaUser?: string | undefined; userIp?: string | undefined; resource: CreateOrdersRequest; }'.

    at /Users/namoscato/dev/git/google-api-typings-generator/node_modules/dtslint/bin/index.js:193:19

Originally posted by @namoscato in #42 (comment)

Somewhat related to #11 but opening a dedicated issue to start discussion.

Some initial thoughts / questions:

1. When should this happen? It would be nice to understand when a Google API changes, which could trigger a build. But any changes to the core generator would ideally regenerate all typings.
2. Would this automatically open DefinitelyTyped PRs? Should we chat with some DefinitelyTyped maintainers before getting too far down this path? Perhaps we could start with just automatically generating the types in a separate branch within this repository? That would probably be a forward-compatible approach to iterate on as well.

Currently we have

process.on('unhandledRejection', reason => {
  throw reason;
});

in order to quit node process with error code on unhandled rejected promises.
Which might not be the best way to do it.
First of all, we have many scripts where we need this and code duplication is not a good thing, at least we should make it includable.

Also, we might want to use --unhandled-rejections=strict option for node instead. But first we have to verify that ts-node supports it, and then add it to package.json scripts, probably.

Investigate this error: https://github.com/Maxim-Mazurok/google-api-typings-generator/runs/578714679?check_suite_focus=true#step:6:1435

It happens sometimes, I can't reproduce it locally, so we need to add more verbose logging.

For starters, mirror shell output to console.

Most likely, it failed during git stash command, but git stash return code 0 when there's no files to stash, so it failed on git pop which returns code 1 when there's no stash to pop.

Google seems to be rolling out API type definitions for Apigee

And it somehow caused our build to fail: https://github.com/Maxim-Mazurok/google-api-typings-generator/runs/517518900?check_suite_focus=true#step:5:351

Further investigation required. I have API changes log on server (#25 script still runs on my server, apparently)

Here's the most recent Apigee JSON: https://gist.github.com/Maxim-Mazurok/acff416616aa0b48da002e0e823c9d43

It would be neat to add some additional context to this in the description that notes the revision change or something, but that could certainly be an iteration.

Thoughts on making this a little more readable, i.e.

[gapi.client.sheets] Update Google Sheets API

On second thought, I suppose it might be a little easier said than done to extract the readable name. Perhaps another iteration as well.

Originally posted by @namoscato in #74

Useful for running in corporate environments

Don't commit changes from other types.
Our update mechanism doesn't work correctly. It treats branch updates to master as new commits which is not good.

Example: DefinitelyTyped/DefinitelyTyped#43734
When merging, it only affects our files, but commits contain modifications in other types, which confuses DT bot: DefinitelyTyped/DefinitelyTyped#43734 (comment)

Another example: DefinitelyTyped/DefinitelyTyped@master...Maxim-Mazurok:gapi.client.sheets
It should not show changes in other types. And our branches should only be ahead of master, not 4 commits ahead, 23 commits behind DefinitelyTyped:master.

Noticed this error in our CI lint action:

Generating playcustomapp:v1 definitions... 

/home/runner/work/google-api-typings-generator/google-api-typings-generator/src/app.ts:666
          const api = JSON.parse(body) as gapi.client.discovery.DirectoryList;
                           ^
SyntaxError: Unexpected token � in JSON at position 0

Full log: https://github.com/Maxim-Mazurok/google-api-typings-generator/pull/35/checks?check_run_id=388279155#step:4:134

Hard to tell what went wrong because we don't log JSON in case of error.
I will open PR to log response body in case of JSON parsing error.

I think typescript-eslint is the preferred option nowadays with TSLint being deprecated?

What about publishing the typings to the repository's GitHub Packages?

From the documentation:

By default, GitHub Packages publishes a package in the GitHub repository you specify in the name field of the package.json file. For example, you would publish a package named @my-org/test to the my-org/test GitHub repository.

Originally posted by @EdricChan03 in #85 (comment)

I think we can do this for "nightly" builds, that will be published without any manual reviews by DefinetlyTyped team.

TODO:

• Update PR line regarding using nightly types: #74 (comment)

Do we have to suppress linter rules for interface Iexample or only for interface IExample?

At least:

• readme.dot
• tsconfig.dot
• tslint.dot

are in the root directory, which is confusing as they are not configuration for this repository.

Move these somewhere else for clarity – perhaps src/templates?

Discovery API returns methods not in the same order, so I had to use deep-sort-object before comparing old and new API version. I think we should also use it (or some similar) approach for typings generation as well, so that we won't get tons of false-changes.

Originally posted by @Maxim-Mazurok in #15 (comment)

Per DefinitelyTyped/DefinitelyTyped#40407 (comment), we might consider decreasing the generated TypeScript version. It would be cool to add some CI to ensure the generated types are compatible across versions.

See I want to use features from TypeScript 2.9 or above.

After running npm start and generating all typings, I found potential flacky bug / heisenbug

At the end of generated types/gapi.client.sql/index.d.ts there was extra } that caused tests to fail:

...
declare namespace gapi.client {
  ...
}

} // <-- this is error

After running npx ts-node -T src/cli.ts --out ./types --service sql typings were generated correctly.

And I couldn't repeat this bug later.

So, I'm documenting this issue in case if this bug reappears.

As we can see here: DefinitelyTyped/DefinitelyTyped#44096
Script force-pushes to DT fork branches every hour, which triggers CI checks and floods PR log for no good reason.
We should check if this push will change anything before doing so.

Please, update PR template, current SHA 5e283ccee07a788a471cfd554b845215de74a549 doesn't match actual SHA b009fe3dc1ccc1067842f50dba7222fcb1cc6b96

I've been using Dependabot for a lot of my projects recently. Thoughts on configuring this project to use it?

Please, update PR template, current SHA ab3f84d44f18d1f33cc79c5d857067b939813c7e doesn't match actual SHA ab3f84d44f18d1f33cc79c5d857067b939813c7e

In branchesDiffer() function, when comparing published branch and local - local may contain other unrelated to the current type changes. It's bad because it will think that something changed and force-push, while the actual type folder doesn't have changes.
This should be fixed.

Learning more about expression syntax, there is probably an option to minimize some of this duplication, consolidating workflows in favor of conditional logic, but this is duplicated for now.

Originally posted by @namoscato in #98

Fix this:

ERROR: 1:1   npm-naming              Declaration file must have a matching npm package.
To resolve this error, either:
1. Change the name to match an npm package.
2. Add a Definitely Typed header with the first line


// Type definitions for non-npm package gapi.client.tasks-browser

Add -browser to the end of your name to make sure it doesn't conflict with existing npm packages.
If you won't fix this error now or you think this error is wrong,
you can disable this check by adding the following options to your project's tslint.json file under "rules":

    "npm-naming": false
 See: https://github.com/Microsoft/dtslint/blob/master/docs/npm-naming.md
ERROR: 2:12  no-trailing-whitespace  trailing whitespace
ERROR: 2:13  dt-header               Error parsing header. Expected: /[^,\r\n]+/. See: https://github.com/Microsoft/dtslint/blob/master/docs/dt-header.md

https://github.com/Maxim-Mazurok/google-api-typings-generator/runs/695930908?check_suite_focus=true#step:9:677

Let's add a new comment line, like this:

// Generated from: https://sheets.googleapis.com/$discovery/rest?version=v4
// Revision: 20191204

And we could easily parse it using /^\/\/ Revision: (\d{8})$/gm

Originally posted by @Maxim-Mazurok in #15 (comment)

@namoscato and @declanvong should be listed in the types owners header as well.

Also, I will be happy to add @harpojaeger once he contributes to this project.

Let me know what you guys think and open PR to add yourself. Change should be made somewhere here:

I appreciate your collaboration!

Make sure that all files (README.md especially) use LF EOL instead of CRLF.

Currently, the type definitions are automatically deployed to the types branch. However, there's no documentation that indicates how to use the generated type definitions in a project.

namoscato:ci...ci-lint prototypes build / linting CI, and here is an example run which is currently failing.

Thoughts on this approach and how it could relate to generating these type definitions automatically longterm?

Related to #206