The fetch function seems to ignore the .ok status of the fetch response. This means that if there is a gRPC error, data is returned of the error type instead of the declared response type.

Shouldn’t the fetch function throw in case of http errors (I.e. if .ok is false), so the client can catch errors? (And so there is no typing inconsistency)

That breaks streaming requests on Firefox browsers.
Probably need to define polyfill for that.

As of protobuf 3.15 optional fields are available, but they are not supported by the generator:

test/optional/v1/has_optional_field.proto: is a proto3 file that contains optional fields, but code generator protoc-gen-grpc-gateway-ts hasn't been updated to support optional fields in proto3. Please ask the owner of this code generator to support proto3 optional.--grpc-gateway-ts_out:

test/optional/v1/has_optional_field.proto contents:

syntax = "proto3";

package test.optional.v1;

message HasOptionalField {
  string not_optional = 1;
  optional string not_required = 2;
}

If a protocol has messages that have bytes type, generator outputs them as Uint8Array, which is not properly encoded by simple JSON.stringify call. See:

I guess it should be converted to base64 encoded string instead.
For that case stringify calls should either contain replacer or Uint8Array should have toJSON definition.

Context

I am using some messages in my protos tha use the well-known google.protobuf.Timestamp:

message Timestamp {
  // Represents seconds of UTC time since Unix epoch
  // 1970-01-01T00:00:00Z. Must be from 0001-01-01T00:00:00Z to
  // 9999-12-31T23:59:59Z inclusive.
  int64 seconds = 1;

  // Non-negative fractions of a second at nanosecond resolution. Negative
  // second values with fractions must still have non-negative nanos values
  // that count forward in time. Must be from 0 to 999,999,999
  // inclusive.
  int32 nanos = 2;
}

Issue

When using grpc-gateway + grpc-gateway-ts to generate both the Go server and the TS client respectively, the following type is generated:

export type Timestamp = {
  seconds?: string
  nanos?: number
}

However, the Go server encodes the timestamp as a string! (checked the received body in the network tab)

For now, I am manually replacing the generated timestamp.pb.ts for the following:

export type Timestamp = string

I am not entirely sure if the best solution is to fix the client side or the server side in this case...

Description

When a protobuf service consists of several files, and some files have special chars in their names like . or -, the generated TS code has invalid import instructions.

Affected version

https://github.com/grpc-ecosystem/protoc-gen-grpc-gateway-ts/releases/tag/v1.1.2

Reproduce steps

• Prepare the env

git clone [email protected]:grpc-ecosystem/protoc-gen-grpc-gateway-ts.git && \
  cd protoc-gen-grpc-gateway-ts && \
  git checkout v1.1.2 && \
  go install . && \
  mkdir -p tmp/generated && \
  cd tmp

• Create a new proto service with files files service.proto and product-models.proto.

// src: ./service.proto
syntax = "proto3";

package myservice;

import "google/api/annotations.proto";

import "product-models.proto";

service MyService {
  rpc GetProducts(GetProductReuest) returns (GetProductResponse) {
    option (google.api.http) = {
      post: "/products"
      body: "*"
    };
  }
}

message GetProductReuest {
  repeated int64 ids = 1;
}

message GetProductResponse {
  repeated Product products = 1;
}

// src: ./product-models.proto
syntax = "proto3";

package myservice;

message Product {
  string name = 1;
  string description = 2;
}

• Generate TS

protoc --grpc-gateway-ts_out=./generated \
  --grpc-gateway-ts_opt=use_proto_names=true,logtostderr=true,loglevel=debug \
  -I . \
  -I ../integration_tests \
  *.proto

• Observe invalid import name MyserviceProduct-models in tmp/generated/service.pb.ts:

/* eslint-disable */
// @ts-nocheck
/*
* This file is a generated Typescript file for GRPC Gateway, DO NOT MODIFY
*/

import * as fm from "./fetch.pb"
import * as MyserviceProduct-models from "./product-models.pb"
export type GetProductReuest = {
  ids?: string[]
}

export type GetProductResponse = {
  products?: MyserviceProduct-models.Product[]
}

export class MyService {
  static GetProducts(req: GetProductReuest, initReq?: fm.InitReq): Promise<GetProductResponse> {
    return fm.fetchReq<GetProductReuest, GetProductResponse>(`/products`, {...initReq, method: "POST", body: JSON.stringify(req, fm.replacer)})
  }
}

Client side streaming support, in particular when used with bidirectional RPC's, allow for a significant reduction in boilerplate for interactive systems, i.e. a chat server.

Are there plans to add client side streaming at any point?

Thanks!

macos 12.3.1:

$ go install github.com/grpc-ecosystem/[email protected]
# golang.org/x/sys/unix
../../../../go/pkg/mod/golang.org/x/[email protected]/unix/syscall_darwin.1_13.go:25:3: //go:linkname must refer to declared function or variable
../../../../go/pkg/mod/golang.org/x/[email protected]/unix/zsyscall_darwin_amd64.1_13.go:27:3: //go:linkname must refer to declared function or variable
../../../../go/pkg/mod/golang.org/x/[email protected]/unix/zsyscall_darwin_amd64.1_13.go:40:3: //go:linkname must refer to declared function or variable
../../../../go/pkg/mod/golang.org/x/[email protected]/unix/zsyscall_darwin_amd64.go:28:3: //go:linkname must refer to declared function or variable
../../../../go/pkg/mod/golang.org/x/[email protected]/unix/zsyscall_darwin_amd64.go:43:3: //go:linkname must refer to declared function or variable
../../../../go/pkg/mod/golang.org/x/[email protected]/unix/zsyscall_darwin_amd64.go:59:3: //go:linkname must refer to declared function or variable
../../../../go/pkg/mod/golang.org/x/[email protected]/unix/zsyscall_darwin_amd64.go:75:3: //go:linkname must refer to declared function or variable
../../../../go/pkg/mod/golang.org/x/[email protected]/unix/zsyscall_darwin_amd64.go:90:3: //go:linkname must refer to declared function or variable
../../../../go/pkg/mod/golang.org/x/[email protected]/unix/zsyscall_darwin_amd64.go:105:3: //go:linkname must refer to declared function or variable
../../../../go/pkg/mod/golang.org/x/[email protected]/unix/zsyscall_darwin_amd64.go:121:3: //go:linkname must refer to declared function or variable
../../../../go/pkg/mod/golang.org/x/[email protected]/unix/zsyscall_darwin_amd64.go:121:3: too many errors

If open a PR with the following should solve this problem?

go get -u golang.org/x/sys

When using this library on Windows you get generated TS with back-slashes instead of forward-slashes in import statements. Obviously, TS is not very happy about it, and it doesn't work unless you replace all the slashes. Here's an example:

import * as fm from ".\fetch.pb"
import * as GoogleApiHttpbody from ".\google\api\httpbody.pb"
import * as GoogleProtobufEmpty from ".\google\protobuf\empty.pb"
import * as GoogleProtobufTimestamp from ".\google\protobuf\timestamp.pb"

When the message filed contains number it's name in the generated typescript is not consistent with the JSON Mapping done by the grpc-gateway runtime. See example below.

Example input message:

syntax = "proto3";

package example.v1;

message Example { string k8s_field = 1; }

Result:

/* eslint-disable */
// @ts-nocheck
/*
* This file is a generated Typescript file for GRPC Gateway, DO NOT MODIFY
*/
export type Example = {
  k8SField?: string
}

Expected result

export type Example = {
  k8sField?: string
}

Generator uses fieldName function that effectively uses strcase.ToLoweCamel that also uppers the case of characters after any number.

Given a proto file with the following annotation and fieldname for GetUserRequest:

service UserService {
  rpc GetUser(GetUserRequest) returns (GetUserResponse) {
    option (google.api.http) = {
      get: "/api/v1/user/{user_id}"
    };
  }
}

message GetUserRequest {
  string user_id = 1;
}

message GetUserResponse {
  User user = 1;
}

message User {
  string id   = 1;
  string name = 2;
}

generates the following TypeScript:

/*
* This file is a generated Typescript file for GRPC Gateway, DO NOT MODIFY
*/

export type GetUserRequest = {
  userId?: string
}

export type GetUserResponse = {
  user?: User
}

export type User = {
  id?: string
  name?: string
}

export class UserService {
  static GetUser(req: GetUserRequest, initReq?: fm.InitReq): Promise<GetUserResponse> {
    return fm.fetchReq<GetUserRequest, GetUserResponse>(`/api/v1/user/${req["user_id"]}`, {...initReq, method: "GET"})
  }
}

Which throws a type error because of /api/v1/user/${req["user_id"]} as user_id does not exist in the generated type definition for GetUserRequest

I have fixed this locally. Is it ok to create a PR?

If a field is deprecated in the proto definitions, such as

message MyMessage {
  string field_a = 1;
  string field_b = 2 [deprecated = true];
}

Would it be possible to add a deprecated annotation to the generated types:

export type MyMessage = {
  fieldA?: string
  /**
   * @deprecated
   */
  fieldB?: string[]
}

If it did, the typescript compiler picks this up giving a warning:

This is useful when deprecating an old field (and often moving to a new one) in a large codebase as it gives a visual indication where the old one is used.

I don't see this plugins published on https://buf.build, can we publish this so it'll be easier to manage it?

The fix for Issue #22 works only if the bytes are in the body of the request, not in query parameters. In this case, they're being removed by flattenRequestPayload even though the type generated is Uint8Array.

Changing the flattenRequestPayload function to below (see the added third if statement) seems to solve the problem. It reuses the replacer function behaviour.

function flattenRequestPayload<T extends RequestPayload>(
  requestPayload: T,
  path: string = ''
): FlattenedRequestPayload {
  return Object.keys(requestPayload).reduce((acc: T, key: string): T => {
    const value = requestPayload[key];
    const newPath = path ? [path, key].join('.') : key;

    const isNonEmptyPrimitiveArray =
      Array.isArray(value) &&
      value.every(v => isPrimitive(v)) &&
      value.length > 0;

    const isNonZeroValuePrimitive =
      isPrimitive(value) && !isZeroValuePrimitive(value as Primitive);

    let objectToMerge = {};

    if (isPlainObject(value)) {
      objectToMerge = flattenRequestPayload(value as RequestPayload, newPath);
    } else if (isNonZeroValuePrimitive || isNonEmptyPrimitiveArray) {
      objectToMerge = { [newPath]: value };
    } else if (value && value.constructor === Uint8Array) {
      objectToMerge = { [newPath]: b64Encode(value, 0, value.length) };
    }

    return { ...acc, ...objectToMerge };
  }, {} as T) as FlattenedRequestPayload;
}

Given the following proto message:

syntax = "proto3";

import "google/protobuf/timestamp.proto";
import "google/protobuf/any.proto";

message User {
  string                    id       = 1;
  string                    name     = 2;
  google.protobuf.Timestamp my_field = 3;
}

// https://developers.google.com/protocol-buffers/docs/proto3#any
message ErrorStatus {
  string   message                     = 1;
  repeated google.protobuf.Any details = 2;
}

The following TypeScript is being generated which throws a type error because neither timestamp.pb.ts nor any.pb exist.

/*
* This file is a generated Typescript file for GRPC Gateway, DO NOT MODIFY
*/

import * as GoogleProtobufAny from "../google/protobuf/any.pb"
import * as GoogleProtobufTimestamp from "../google/protobuf/timestamp.pb"

export type User = {
  id?: string
  name?: string
  myField?: GoogleProtobufTimestamp.Timestamp
}

export type ErrorStatus = {
  message?: string
  details?: GoogleProtobufAny.Any[]
}

Please let me know if I am missing any steps in the generation process 🙏🏼
I used the following command to generate the above TypeScript file.

protoc -I. --grpc-gateway-ts_out=. --grpc-gateway-ts_opt logtostderr=true --grpc-gateway-ts_opt loglevel=debug ./proto/example.proto

The parser and generator should add support for methods with additional_bindings, allowing multiple different URLs to access the same gRPC endpoint. The current code only renders code for the outer method, but silently ignores the additional_bindings.

This is especially useful when you have bindings using different HTTP verbs, like PUT and PATCH, that hit the same gRPC method but may use different verbs, or populate different fields from the URL.

I have a request message with google.protobuf.BoolValue name = 1 as a field. When I define a value for this field in Typescript like name: { value: false }, then I see that the request made by the generated code does not include the parameter for this field.

It appears that renderURLSearchParams strips out all primitive types with zero values, but that's not expected for BoolValue, since the whole point is to be able to differentiate between false and nil.

Is there an established way to fix issues like this within this library? I see #25 and cresta#1, but it doesn't include BoolValue or the other wrapper well-known types.

It is useful to pass a header like Authorization to the query, I can't find a way to pass it

When a proto file has imports, the generator generates code like

import * as GoogleProtobufTimestamp from "./google/protobuf/timestamp.pb";

The default setup of TypeScript in the bundler I use is strict, and is set up to give errors on type-only imports:

error TS1371: This import is never used as a value and must use 'import type' because 'importsNotUsedAsValues' is set to 'error'.

I can work around this by loosening the tsconfig setting.

Would it be possible to replace the imports generated for these kinds of import by import type to avoid these TypeScript errors on strict setups?

Hi, I am trying to setup grpc-gateway for the service and doing it using Google AIP recommendations and provided linter.

So if you would take a look at https://google.aip.dev/127 - the recommended way of using http annotations for more or less complicated APIs looks like that:

post: "/v1/{parent=publishers/*}/books"

However protoc-gen-grpc-gateway-ts would not generate expected for that case /v1/${req["parent"]}/books, instead of that protoc-gen-grpc-gateway-ts would generate /v1/${req["parentpublishers"]}/books

Previously discussed in #4

Although we should run most other protoc plugins with WKT definitions come with the protoc (timestamp.pb, duration.pb etc.).
In the case of protoc-gen-grpc-gateway-ts, we should generate TypeScript definitions compatible with grpc-gateway's JSON mapping (and protoc-gen-openapiv2).

Currently, by running the protoc-gen-grpc-gateway-ts on timestamp.pb, we will get the underlying definition of the timestamp.

// example message
export type Message = {
  value?: string
  waitPeriod?: GoogleProtobufDuration.Duration
  ts?: GoogleProtobufTimestamp.Timestamp
}
// the definition of GoogleProtobufTimestamp.Timestamp
export type Timestamp = {
  seconds?: string
  nanos?: number
}

Which will be rejected by grpc-gateway.
The correct JSON mapping of the timestamp is a datetime in RFC3339(nano) format, which can be fulfilled by the default behavior of JSON.stringify a Date object in JavaScript.

I think we still need to set up exception rules for parsing those WKTs, at least for Timestamp and Duration.

atreya2011@a1e3d5c seems to be a good patch without the handling of wrappers, duration and etc.

The following is extracted from this file: http.proto

// HTTP | gRPC
// -----|-----
// `GET /v1/messages/123456`  | `GetMessage(name: "messages/123456")`
//
// Any fields in the request message which are not bound by the path template
// automatically become HTTP query parameters if there is no HTTP request body.
// For example:
//
//     service Messaging {
//       rpc GetMessage(GetMessageRequest) returns (Message) {
//         option (google.api.http) = {
//             get:"/v1/messages/{message_id}"
//         };
//       }
//     }
//     message GetMessageRequest {
//       message SubMessage {
//         string subfield = 1;
//       }
//       string message_id = 1; // Mapped to URL path.
//       int64 revision = 2;    // Mapped to URL query parameter `revision`.
//       SubMessage sub = 3;    // Mapped to URL query parameter `sub.subfield`.
//     }
//
// This enables a HTTP JSON to RPC mapping as below:
//
// HTTP | gRPC
// -----|-----
// `GET /v1/messages/123456?revision=2&sub.subfield=foo` |
// `GetMessage(message_id: "123456" revision: 2 sub: SubMessage(subfield:
// "foo"))`
//

However it seems like renderURL function in protoc-gen-grpc-gateway doesn't render the fields in the request message which are not bound by the path template to become URL query parameters if there is no HTTP request body.

Any advice on how to handle this would be great! Thanks in advance 🙏🏼

protoc-gen-grpc-gateway-ts always put ? suffix for the fields, as fields in proto3 are basically optional. But I want the generator to support omitting the ? suffix when it's marked as non-optional in a way.

An idea is to consider a field is a required and If the field is specified as google.api.field_behavior = REQUIRED annotation (cf. https://github.com/googleapis/googleapis/blob/master/google/api/field_behavior.proto#L61 and an example https://github.com/googleapis/googleapis/blob/master/google/logging/v2/logging.proto#L154).

I think it is supported in the reverse proxy.

I'd like to use messagepack as the transport codec. So far I did not find a way to achieve this.
The Server implementation of grpc gateway allows to define a custom marhaller. Such a solution in this lib would be great.