googleapis / java-core Goto Github PK

A set of classes and utilities used in Google Cloud Java libraries

License: Apache License 2.0

java-core's Introduction

Google Cloud Java Client -- Core

A set of classes and utilities used in Google Cloud Java libraries.

🚌 In January 2023, this library has moved to sdk-platform-java/java-core. This repository will be archived in the future. Future releases will appear in the new repository (https://github.com/googleapis/google-cloud-java/releases). The Maven artifact coordinates (such as com.google.cloud:google-cloud-core) remain the same.

Note: This library is only meant to be consumed by other Google Libraries.

API Documentation

Java Versions

Java 7 or above is required for using this client.

Contributing

Contributions to this library are always welcome and highly encouraged.

See CONTRIBUTING documentation for more information on how to get started.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See Code of Conduct for more information.

Versioning

This library follows Semantic Versioning.

It is currently in major version two (2.y.z), which means that the public API should be considered stable.

License

Apache 2.0 - See LICENSE for more information.

CI Status

Java Version	Status
Java 7
Java 8
Java 8 OSX
Java 8 Windows
Java 11

java-core's People

Contributors

Stargazers

Watchers

java-core's Issues

Confusing documentation for com.google.cloud.Timestamp.parseTimestamp

The Javadoc for com.google.cloud.Timestamp.parseTimestamp says that the input doesn't have a timezone offset and ends with "Z":

java-core/google-cloud-core/src/main/java/com/google/cloud/Timestamp.java

Lines 191 to 195 in 1ba0266

 /** 

  * Creates a Timestamp instance from the given string. String is in the RFC 3339 format without 

  * the timezone offset (always ends in "Z"). 

  */ 

 public static Timestamp parseTimestamp(String timestamp) {

However, it seems to parse offsets correctly with google-cloud-core 1.93.7:

import com.google.cloud.Timestamp;
import org.junit.Test;

public class TimestampTest {

  @Test
  public void parseTimestamp() {
    System.err.println(Timestamp.parseTimestamp("2020-07-10T14:03:00-07:00"));
    System.err.println(Timestamp.parseTimestamp("2020-07-10T14:03:00Z"));
  }
}

2020-07-10T21:03:00Z
2020-07-10T14:03:00Z

I would expect either of the following:

parseTimestamp handles a timezone offset, and the documentation doesn't specify that there is no offset.
parseTimestamp doesn't handle a timezone offset and throws an exception when one is present.

This issue may be related to googleapis/google-cloud-java#4583. The fix changed the parsing logic but not the documentation.

Include LICENSE in jar files

Is your feature request related to a problem? Please describe.
The jar artifacts produced out of this repository do not include the LICENSE file.

Describe the solution you'd like
Include LICENSE in any built jar artifacts.

Describe alternatives you've considered
Manual overrides for our license detection/handling code, for Google Cloud libraries.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

These problems occurred while renovating this repository.

WARN: RepoCacheS3.getCacheFolder() - appending missing trailing slash to pathname

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

deps: update actions/checkout action to v3

Detected dependencies

github-actions

.github/workflows/approve-readme.yaml

actions/github-script v6

.github/workflows/auto-release.yaml

actions/github-script v6

.github/workflows/ci.yaml

actions/checkout v3

actions/setup-java v3

actions/checkout v3

actions/setup-java v3

actions/checkout v3

actions/setup-java v3

actions/checkout v3

actions/setup-java v3

actions/checkout v3

actions/setup-java v3

.github/workflows/downstream.yaml

actions/checkout v2

actions/setup-java v3

maven

google-cloud-core-bom/pom.xml

com.google.cloud:google-cloud-shared-config 1.5.5

com.google.cloud:google-cloud-core 2.9.5-SNAPSHOT

com.google.cloud:google-cloud-core-grpc 2.9.5-SNAPSHOT

com.google.cloud:google-cloud-core-http 2.9.5-SNAPSHOT

google-cloud-core-grpc/pom.xml

com.google.cloud:google-cloud-core-parent 2.9.5-SNAPSHOT

google-cloud-core-http/pom.xml

com.google.cloud:google-cloud-core-parent 2.9.5-SNAPSHOT

google-cloud-core/pom.xml

com.google.cloud:google-cloud-core-parent 2.9.5-SNAPSHOT

pom.xml

com.google.cloud:google-cloud-shared-config 1.5.5

com.google.auth:google-auth-library-bom 1.14.0

com.google.api:gax-bom 2.22.0

com.google.http-client:google-http-client-bom 1.42.3

com.google.api-client:google-api-client-bom 2.1.2

io.grpc:grpc-bom 1.52.1

com.google.protobuf:protobuf-bom 3.21.12

com.google.guava:guava-bom 31.1-jre

com.google.api:api-common 2.5.0

com.google.api.grpc:proto-google-common-protos 2.13.0

com.google.api.grpc:proto-google-iam-v1 1.8.0

io.opencensus:opencensus-api 0.31.1

io.opencensus:opencensus-contrib-http-util 0.31.1

javax.annotation:javax.annotation-api 1.3.2

com.google.code.findbugs:jsr305 3.0.2

org.threeten:threetenbp 1.6.5

com.google.errorprone:error_prone_annotations 2.18.0

com.google.code.gson:gson 2.10.1

com.google.truth:truth 1.1.3

junit:junit 4.13.2

org.easymock:easymock 5.1.0

org.objenesis:objenesis 3.3

org.apache.maven.plugins:maven-project-info-reports-plugin 3.4.2

org.apache.maven.plugins:maven-javadoc-plugin 3.4.1

Check this box to trigger a request for Renovate to run again on this repository

Policy equality fails for binding list order

The binding list order should not matter for the equivalence of Policy objects:

  @Test
  public void testPolicyOrderShouldNotMatter() {
    Role role1 = Role.of("role1");
    Identity identity1 = Identity.user("[email protected]");
    Role role2 = Role.of("role2");
    Identity identity2 = Identity.user("[email protected]");
    Policy policy1 = Policy.newBuilder()
            .addIdentity(role1, identity1)
            .addIdentity(role2, identity2)
            .build();
    Policy policy2 = Policy.newBuilder()
            .addIdentity(role2, identity2)
            .addIdentity(role1, identity1)
            .build();
    assertEquals(policy1, policy2);
  }

Synthesis failed for java-core

Hello! Autosynth couldn't regenerate java-core. 💔

Here's the output from running synth.py:

Cloning into 'working_repo'...
Switched to branch 'autosynth'
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.1/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.1/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/git/autosynth/autosynth/synth.py", line 256, in <module>
    main()
  File "/tmpfs/src/git/autosynth/autosynth/synth.py", line 196, in main
    last_synth_commit_hash = get_last_metadata_commit(args.metadata_path)
  File "/tmpfs/src/git/autosynth/autosynth/synth.py", line 149, in get_last_metadata_commit
    text=True,
  File "/home/kbuilder/.pyenv/versions/3.6.1/lib/python3.6/subprocess.py", line 403, in run
    with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'text'

Google internal developers can see the full log here.

BaseEmulatorHelper prefers bin emulator runner when gcloud is available

Configuration

google-cloud-core:1.93.9
Google Cloud SDK 327.0.0

Problem

I noticed that having installed gcloud this class prefers bin emulator against a google one. I would prefer to start the latter when I have, cause no external dependencies.

The method com.google.cloud.testing.BaseEmulatorHelper.GcloudEmulatorRunner #isGcloudInstalled checks that google-cloud-sdk contains in PATH, but for me there is no one, as I have dynamic configuration, i.e. with brew.

My PATH: /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

Is it possible to change the check to the side of something like gcloud --version? The current check isn't solid

Configuration error for release-please

JavaYoshi (googleapis/java-core): Missing required file: versions.txt

Core: BaseEmulatorHelper fails to detect GCloud installation on Windows

Environment details

API: Core
OS type and version: Windows 10
Java version: 11
google-cloud-core version(s): 1.94.1

Steps to reproduce

Run an emulator on Windows. For example, using LocalDatastoreHelper
The GCloud installation is not detected, so the emulator runs from a downloaded zip file

Code example

LocalDatastoreHelper datastoreHelper = LocalDatastoreHelper.create();
datastoreHelper.start();

Look at the processes started. It will not be the gcloud-based variant, even if gcloud is installed. You can tell it's the gcloud variant if a python process is started.

Stack trace

N/A

External references such as API reference guides

Any additional information below

In Windows the PATH environment variable is Path (not all uppercase) so this is why it is not detecting the installation.

Timestamp ofTimeMicroseconds(long microseconds) fails for epoch dates with microseconds

com.google.cloud.Timestamp.ofTimeMicroseconds(long microseconds) fails with IllegalArgumentException if the given date is epoch and not truncated to seconds, in the concrete example, -8499613106649 epoch timestamp was used.

google-cloud-core version: 1.62.0

Stacktrace fragment:

java.lang.IllegalArgumentException: timestamp out of range: -8499613106649, 749440000
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:357)
at com.google.cloud.Timestamp.ofTimeMicroseconds(Timestamp.java:96)

Synthesis failed for java-core

Hello! Autosynth couldn't regenerate java-core. 💔

Here's the output from running synth.py:

Cloning into 'working_repo'...
Switched to a new branch 'autosynth'
Cloning into '/tmpfs/tmp/tmp126vhjo7/synthtool'...
Switched to branch 'autosynth-self'
Note: checking out 'e56b3ef0e2ea5c96b36f8aae851fd26efe1337c1'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at e56b3ef deps: update dependency com.google.http-client:google-http-client-bom to v1.35.0 (#211)
Note: checking out '52638600f387deb98efb5f9c85fec39e82aa9052'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 5263860 build(java): set GOOGLE_CLOUD_PROJECT env for samples/integration tests (#484)
Switched to a new branch 'autosynth-self-1'
2020-04-30 14:02:34 [INFO] Running synthtool
2020-04-30 14:02:34 [INFO] ['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 142, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "/tmpfs/src/github/synthtool/synthtool/__init__.py", line 21, in <module>
    from synthtool import update_check
  File "/tmpfs/src/github/synthtool/synthtool/update_check.py", line 19, in <module>
    import packaging.version
ModuleNotFoundError: No module named 'packaging'
2020-04-30 14:02:34 [ERROR] Synthesis failed
HEAD is now at e56b3ef deps: update dependency com.google.http-client:google-http-client-bom to v1.35.0 (#211)
Switched to branch 'autosynth-self'
Note: checking out 'e56b3ef0e2ea5c96b36f8aae851fd26efe1337c1'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at e56b3ef deps: update dependency com.google.http-client:google-http-client-bom to v1.35.0 (#211)
Previous HEAD position was 5263860 build(java): set GOOGLE_CLOUD_PROJECT env for samples/integration tests (#484)
HEAD is now at 6b685a2 fix: synthtool path (#515)
Switched to a new branch 'autosynth-16'
2020-04-30 14:02:34 [INFO] Running synthtool
2020-04-30 14:02:34 [INFO] ['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 142, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "/tmpfs/src/github/synthtool/synthtool/__init__.py", line 21, in <module>
    from synthtool import update_check
  File "/tmpfs/src/github/synthtool/synthtool/update_check.py", line 19, in <module>
    import packaging.version
ModuleNotFoundError: No module named 'packaging'
2020-04-30 14:02:34 [ERROR] Synthesis failed
HEAD is now at e56b3ef deps: update dependency com.google.http-client:google-http-client-bom to v1.35.0 (#211)
Switched to branch 'autosynth'
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 576, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 457, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 566, in _inner_main
    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 352, in synthesize_loop
    synthesize_inner_loop(toolbox, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 362, in synthesize_inner_loop
    synthesizer, len(toolbox.versions) - 1
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 259, in synthesize_version_in_new_branch
    synthesizer.synthesize(self.environ)
  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 115, in synthesize
    synth_proc.check_returncode()  # Raise an exception.
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode
    self.stderr)
subprocess.CalledProcessError: Command '['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']' returned non-zero exit status 1.

Google internal developers can see the full log here.

Warning: a recent release failed

The following release PRs may have failed:

#1002 - The release job was triggered, but has not reported back success.

Synthesis failed for java-core

Hello! Autosynth couldn't regenerate java-core. 💔

Please investigate and fix this issue within 5 business days. While it remains broken,
this library cannot be updated with changes to the java-core API, and the library grows
stale.

See https://github.com/googleapis/synthtool/blob/master/autosynth/TroubleShooting.md
for trouble shooting tips.

Here's the output from running synth.py:

t checkout 9f26800e9c531ba0b4f9f9bde0786f72ef14c0df
Note: switching to '9f26800e9c531ba0b4f9f9bde0786f72ef14c0df'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 9f26800 chore: release 1.95.5-SNAPSHOT (#481)
2021-07-01 01:34:17,639 autosynth [DEBUG] > Running: git checkout 87254ac89a9559864c0a245d6b150406439ce3d8
Previous HEAD position was 333fd90 chore: update precommit hook pre-commit/pre-commit-hooks to v4 (#1083)
HEAD is now at 87254ac fix: Add `shopt -s nullglob` to dependencies script (#1130)
2021-07-01 01:34:17,648 autosynth [DEBUG] > Running: git branch -f autosynth-26
2021-07-01 01:34:17,651 autosynth [DEBUG] > Running: git checkout autosynth-26
Switched to branch 'autosynth-26'
2021-07-01 01:34:17,656 autosynth [INFO] > Running synthtool
2021-07-01 01:34:17,656 autosynth [INFO] > ['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']
2021-07-01 01:34:17,656 autosynth [DEBUG] > log_file_path: /tmpfs/src/logs/java-core/26/sponge_log.log
2021-07-01 01:34:17,658 autosynth [DEBUG] > Running: /tmpfs/src/github/synthtool/env/bin/python3 -m synthtool --metadata synth.metadata synth.py --
2021-07-01 01:34:17,893 synthtool [DEBUG] > Executing /home/kbuilder/.cache/synthtool/java-core/synth.py.
On branch autosynth-26
nothing to commit, working tree clean
2021-07-01 01:34:18,156 synthtool [DEBUG] > Using precloned repo /home/kbuilder/.cache/synthtool/synthtool
.github/CODEOWNERS
.github/ISSUE_TEMPLATE/bug_report.md
.github/ISSUE_TEMPLATE/feature_request.md
.github/ISSUE_TEMPLATE/support_request.md
.github/PULL_REQUEST_TEMPLATE.md
.github/blunderbuss.yml
.github/generated-files-bot.yml
.github/readme/synth.py
.github/release-please.yml
.github/snippet-bot.yml
.github/sync-repo-settings.yaml
.github/trusted-contribution.yml
.github/workflows/approve-readme.yaml
.github/workflows/auto-release.yaml
.github/workflows/ci.yaml
.github/workflows/samples.yaml
.kokoro/build.bat
.kokoro/build.sh
.kokoro/coerce_logs.sh
.kokoro/common.cfg
.kokoro/common.sh
.kokoro/continuous/common.cfg
.kokoro/continuous/java8.cfg
.kokoro/continuous/readme.cfg
.kokoro/dependencies.sh
.kokoro/nightly/common.cfg
.kokoro/nightly/integration.cfg
.kokoro/nightly/java11.cfg
.kokoro/nightly/java7.cfg
.kokoro/nightly/java8-osx.cfg
.kokoro/nightly/java8-win.cfg
.kokoro/nightly/java8.cfg
.kokoro/nightly/samples.cfg
.kokoro/populate-secrets.sh
.kokoro/presubmit/clirr.cfg
.kokoro/presubmit/common.cfg
.kokoro/presubmit/dependencies.cfg
.kokoro/presubmit/integration.cfg
.kokoro/presubmit/java11.cfg
.kokoro/presubmit/java7.cfg
.kokoro/presubmit/java8-osx.cfg
.kokoro/presubmit/java8-win.cfg
.kokoro/presubmit/java8.cfg
.kokoro/presubmit/linkage-monitor.cfg
.kokoro/presubmit/lint.cfg
.kokoro/presubmit/samples.cfg
.kokoro/readme.sh
.kokoro/release/bump_snapshot.cfg
.kokoro/release/common.cfg
.kokoro/release/common.sh
.kokoro/release/drop.cfg
.kokoro/release/drop.sh
.kokoro/release/promote.cfg
.kokoro/release/promote.sh
.kokoro/release/publish_javadoc.cfg
.kokoro/release/publish_javadoc.sh
.kokoro/release/publish_javadoc11.cfg
.kokoro/release/publish_javadoc11.sh
.kokoro/release/snapshot.cfg
.kokoro/release/snapshot.sh
.kokoro/release/stage.cfg
.kokoro/release/stage.sh
.kokoro/trampoline.sh
CODE_OF_CONDUCT.md
CONTRIBUTING.md
LICENSE
README.md
SECURITY.md
codecov.yaml
java.header
license-checks.xml
renovate.json
samples/install-without-bom/pom.xml
samples/pom.xml
samples/snapshot/pom.xml
samples/snippets/pom.xml
2021-07-01 01:34:18,345 synthtool [DEBUG] > existing pom file found (.github/sync-repo-settings.yaml) - keeping the existing
2021-07-01 01:34:21,431 synthtool [DEBUG] > Wrote metadata to synth.metadata.
2021-07-01 01:34:21,491 autosynth [INFO] > Changed files:
2021-07-01 01:34:21,491 autosynth [INFO] > M .kokoro/coerce_logs.sh
 M .kokoro/dependencies.sh
 M synth.metadata
2021-07-01 01:34:21,491 autosynth [DEBUG] > Running: git log 87254ac89a9559864c0a245d6b150406439ce3d8 -1 --no-decorate --pretty=%s
2021-07-01 01:34:21,495 autosynth [DEBUG] > Running: git log 87254ac89a9559864c0a245d6b150406439ce3d8 -1 --no-decorate --pretty=%b%n%nSource-Author: %an <%ae>%nSource-Date: %ad
2021-07-01 01:34:21,498 autosynth [DEBUG] > Running: git add -A
2021-07-01 01:34:21,503 autosynth [DEBUG] > Running: git status --porcelain
2021-07-01 01:34:21,509 autosynth [DEBUG] > Running: git commit -m fix: Add `shopt -s nullglob` to dependencies script

This prevents processing the unexpanded glob "**/.flattened-pom.xml" in repos which do not flatten any POMs.

Source-Author: dpcollins-google <[email protected]>
Source-Date: Sat Jun 19 18:35:58 2021 -0400
Source-Repo: googleapis/synthtool
Source-Sha: 87254ac89a9559864c0a245d6b150406439ce3d8
Source-Link: https://github.com/googleapis/synthtool/commit/87254ac89a9559864c0a245d6b150406439ce3d8
[autosynth-26 b80065f] fix: Add `shopt -s nullglob` to dependencies script
 3 files changed, 6 insertions(+), 5 deletions(-)
2021-07-01 01:34:21,516 autosynth [DEBUG] > Running: git reset --hard HEAD
HEAD is now at b80065f fix: Add `shopt -s nullglob` to dependencies script
2021-07-01 01:34:21,522 autosynth [DEBUG] > Running: git checkout autosynth
Switched to branch 'autosynth'
2021-07-01 01:34:21,527 autosynth [DEBUG] > Running: git diff --binary HEAD..autosynth-26 -- . :(exclude)synth.metadata
2021-07-01 01:34:21,531 autosynth [DEBUG] > Running: git diff --binary HEAD autosynth-26
2021-07-01 01:34:21,535 autosynth [DEBUG] > Running: git apply /tmpfs/tmp/tmpizmhicrd/autosynth-26.patch
2021-07-01 01:34:21,538 autosynth [DEBUG] > Running: git add -A
2021-07-01 01:34:21,543 autosynth [DEBUG] > Running: git status --porcelain
2021-07-01 01:34:21,549 autosynth [DEBUG] > Running: git commit -m fix: Add `shopt -s nullglob` to dependencies script

This prevents processing the unexpanded glob "**/.flattened-pom.xml" in repos which do not flatten any POMs.

Source-Author: dpcollins-google <[email protected]>
Source-Date: Sat Jun 19 18:35:58 2021 -0400
Source-Repo: googleapis/synthtool
Source-Sha: 87254ac89a9559864c0a245d6b150406439ce3d8
Source-Link: https://github.com/googleapis/synthtool/commit/87254ac89a9559864c0a245d6b150406439ce3d8
[autosynth 366ae01] fix: Add `shopt -s nullglob` to dependencies script
 2 files changed, 2 insertions(+), 1 deletion(-)
2021-07-01 01:34:21,556 autosynth [DEBUG] > Running: git diff --binary HEAD..autosynth-27 -- . :(exclude)synth.metadata
2021-07-01 01:34:21,560 autosynth [DEBUG] > Running: git diff HEAD..autosynth-27 -- synth.metadata
2021-07-01 01:34:21,564 autosynth [DEBUG] > Running: git diff --binary autosynth-27..autosynth-30 -- . :(exclude)synth.metadata
2021-07-01 01:34:21,567 autosynth [DEBUG] > Running: git diff autosynth-27..autosynth-30 -- synth.metadata
2021-07-01 01:34:21,571 autosynth [DEBUG] > Running: git diff --binary autosynth-30..autosynth-40 -- . :(exclude)synth.metadata
2021-07-01 01:34:21,574 autosynth [DEBUG] > Running: git diff autosynth-30..autosynth-40 -- synth.metadata
2021-07-01 01:34:21,577 autosynth [DEBUG] > Running: git push --force origin autosynth
remote: 
remote: Create a pull request for 'autosynth' on GitHub by visiting:        
remote:      https://github.com/googleapis/java-core/pull/new/autosynth        
remote: 
To https://github.com/googleapis/java-core.git
 * [new branch]      autosynth -> autosynth
2021-07-01 01:34:24,349 autosynth [DEBUG] > Running: git log -3 --pretty=%b
2021-07-01 01:34:27,510 autosynth [DEBUG] > Running: git clean -fdx
Removing __pycache__/
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 356, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 191, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 336, in _inner_main
    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 92, in synthesize_loop
    if change_pusher.check_if_pr_already_exists(fork.branch):
  File "/tmpfs/src/github/synthtool/autosynth/change_pusher.py", line 117, in check_if_pr_already_exists
    prs = self._gh.list_pull_requests(repo, state="open", head=f"{owner}:{branch}")
  File "/tmpfs/src/github/synthtool/autosynth/github.py", line 61, in list_pull_requests
    return cast(List[Dict], _get_json_or_raise_exception(response))
  File "/tmpfs/src/github/synthtool/autosynth/github.py", line 478, in _get_json_or_raise_exception
    json = response.json()
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/requests/models.py", line 900, in json
    return complexjson.loads(self.text, **kwargs)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Google internal developers can see the full log here.

GrpcTransportOptions uses internal grpc APIs

This is reopening #120 which should not have been closed. Internal APIs change without warning, which would break your users. Please remove the usage of the internal APIs.

java-core/google-cloud-core-grpc/src/main/java/com/google/cloud/grpc/GrpcTransportOptions.java

Lines 35 to 36 in 13414b6

 import io.grpc.internal.SharedResourceHolder; 

 import io.grpc.internal.SharedResourceHolder.Resource;

Add

Linkage problem: toImmutableSet

I tried adding the linkage checker enforcer rule to this project and got this error:

[ERROR] Linkage Checker rule found 1 error. Linkage error report:
(guava-28.1-android.jar) com.google.common.collect.ImmutableSet's method toImmutableSet() is not found;
referenced by 1 class file
com.google.auto.service.processor.AutoServiceProcessor (auto-service-1.0-rc4.jar)

Warning: a recent release failed

The following release PRs may have failed:

#863 - The release job was triggered, but has not reported back success.

Support conditional IAM Policies

Introduce conditional IAM Policies into java-core to be used by product libraries such as Cloud Storage.

Design Document.

Warning: a recent release failed

The release PR #158 is still in a pending state after several hours

com.google.cloud.Identity does not support the new IAM deleted user syntax

The new IAM deleted user syntax is not supported by com.google.cloud.Identity:
https://cloud.google.com/iam/docs/policies#handle-deleted-members

Environment details

Affects the Core library and all dependant libraries
OS type and version: any
Java version: any
google-cloud-core version(s): any

Steps to reproduce

Try to parse a deleted user string with Identity.valueOf
This results either to a stacktrace (1.74.0 and lower) or a DELETED identity type (1.75.0 and higher)

Code example

Identity identity = Identity.valueOf("deleted:user:[email protected]?uid=234567890123456789012");
System.out.println("type = " + identity.getType());
System.out.println("value = " + identity.getValue());

Stack trace

1.74.0 and lower

java.lang.IllegalArgumentException: Illegal identity string: "deleted:user:[email protected]?uid=234567890123456789012"

1.75.0 and higher

type = DELETED
value = user:[email protected]?uid=234567890123456789012

The "fix" is caused by an unrelated change: 24667e5

External references such as API reference guides

https://cloud.google.com/iam/docs/policies#handle-deleted-members

Please increase google-oauth due to security issue

We use snyk tool to report security issues in our team.
This tools reported us that lib google-oauth-client used by google-cloud-bigquery has vulnerability. Full snyk info:

Overview
com.google.oauth-client:google-oauth-client is a powerful and easy-to-use Java library for the OAuth 1.0a and OAuth 2.0 authorization standards.

Affected versions of this package are vulnerable to Improper Authorization. PKCE support is not implemented in accordance with the RFC for OAuth 2.0 for Native Apps. Without the use of PKCE, the authorization code returned by an authorization server is not enough to guarantee that the client that issued the initial authorization request is the one that will be authorized. An attacker is able to obtain the authorization code using a malicious app on the client-side and use it to gain authorization to the protected resource.

Remediation
Upgrade com.google.oauth-client:google-oauth-client to version 1.31.0 or higher.

Could you please use newer version of this lib?

Warning: a recent release failed

The release PR #134 is still in a pending state after several hours

Timestamp.of(java.sql.Timestamp) off-by-one on pre-epoch dates

Originally reported by @s13o in spring-attic/spring-cloud-gcp#2041

Increase DEFAULT_CHUNK_SIZE to reduce transfer overhead, increase throughput

Is your feature request related to a problem? Please describe.
In working with GCS customers, I've observed that the Java client library for storage tends to have poor single-threaded, single-stream upload throughput. The issue presented itself a lot like this one, but for uploads instead of downloads. After debugging it for a while, I found that overriding the DEFAULT_CHUNK_SIZE value in the storage WriteChannel with the setChunkSize method to a larger value significantly improved transfer times.

In fact, I put a customer's (idiomatic) code in a test harness and experimented with different chunk sizes to find out if there was an inflection point where larger chunks don't pay off (there is) and what that point might be, as it would seem to be optimal for minimizing memory usage and retry cost and transfer throughput. I can provide raw data upon request, but here's the graph of transfer times of 1GB with different chunk sizes (lower Y value is better).

This test was run from a GCE VM with a same-region Regional bucket. It seems that beyond 15MB, little is gained. Between 1, 2, 5, and 10 MB, a lot is gained. The gain between 10MB and 15MB is smaller, but 15MB is definitely where improvements for throughput stop.

Describe the solution you'd like
Increase the DEFAULT_CHUNK_SIZE to 15MB (or MiB, since that's what it uses) in BaseWriteChannel.

A larger chunk size would also be consistent with the way GSUtil handles uploads to GCS, with 100MB chunks, heavily optimized for transfer speed.

Describe alternatives you've considered
The alternative is the status quo, in which customers should override this value unless they are optimizing heavily for low memory / low retry cost over throughput. I think this optimization is less common than optimization for transfer speed, and regardless, at 10-15MiB a good balance among all three considerations is achieved - especially considering the advancements in mobile computing power and WANs since we last set this value >4 years ago. A balance between likely optimizations is a good place for a default.

Additional context
Disclaimer: I am a Googler, and some amount of discussion on the matter occurred internally, with @frankyn suggesting this approach to me. All the relevant contents of that discussion are presented here.

Use java-shared-dependencies/third-party-dependencies where possible

We should add an import-scoped java-shared-dependencies/third-party-dependencies entry in the dependency-management section of java-core's pom.xml, remove the associated version declarations in java-core, and consider offloading any remaining third-party dependency versions to third-party-dependencies.

Http-based clients should send client version in User-Agent header

GCS reporting can only parse out of the header rather than the x-goog-api-client header.

The format of the User-Agent header should include be gcloud-java/x.y.z

Note: this is a short-term fix

GrpcTransportOptions uses internal grpc APIs

GrpcTransportOptions references io.grpc.internal.SharedResourceHolder. As it is in the internal package, it has an unstable API that may change at any time.

https://github.com/googleapis/google-cloud-java/blob/6b567115e98271632bdaa04833fdd9e217bd9c3e/google-cloud-clients/google-cloud-core-grpc/src/main/java/com/google/cloud/grpc/GrpcTransportOptions.java#L35-L36

Synthesis failed for java-core

Hello! Autosynth couldn't regenerate java-core. 💔

Here's the output from running synth.py:

Cloning into 'working_repo'...
Switched to branch 'autosynth'
Running synthtool
['/tmpfs/src/git/autosynth/env/bin/python3', '-m', 'synthtool', 'synth.py', '--']
synthtool > Executing /tmpfs/src/git/autosynth/working_repo/synth.py.
On branch autosynth
nothing to commit, working tree clean
HEAD detached at FETCH_HEAD
nothing to commit, working tree clean
synthtool > Wrote metadata to synth.metadata.
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.1/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.1/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/git/autosynth/env/lib/python3.6/site-packages/synthtool/__main__.py", line 102, in <module>
    main()
  File "/tmpfs/src/git/autosynth/env/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/tmpfs/src/git/autosynth/env/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/tmpfs/src/git/autosynth/env/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmpfs/src/git/autosynth/env/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/tmpfs/src/git/autosynth/env/lib/python3.6/site-packages/synthtool/__main__.py", line 94, in main
    spec.loader.exec_module(synth_module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
  File "/tmpfs/src/git/autosynth/working_repo/synth.py", line 22, in <module>
    templates = common_templates.java_library()
  File "/tmpfs/src/git/autosynth/env/lib/python3.6/site-packages/synthtool/gcp/common.py", line 75, in java_library
    return self._generic_library("java_library", **kwargs)
  File "/tmpfs/src/git/autosynth/env/lib/python3.6/site-packages/synthtool/gcp/common.py", line 43, in _generic_library
    if not kwargs["metadata"]["samples"]:
KeyError: 'samples'

Synthesis failed

Google internal developers can see the full log here.

Import native-support module as dependency in google-cloud-core

PR #637 adds google-cloud-core-native-support as a separate module. We need to import this module as a dependency for google-cloud-core. This will be a follow-up to #637.

SSLException isn't retryable

Hi,

Storage Java library users are failing on a retryable exception SocketException: Socket closed' errors which raised as a javax.net.ssl.SSLException. SSLException isn't considered retryable in BaseServiceException.java.

Are there any concerns with making this exception retryable along with:

SocketTimeoutException
SocketException
SSLHandshakeException

I'll send a PR to introduce this change for discussion.

Your .repo-metadata.json file has a problem 🤒

You have a problem with your .repo-metadata.json file:

Result of scan 📈:

must have required property 'client_documentation' in .repo-metadata.json

☝️ Once you address these problems, you can close this issue.

Need help?

Schema definition: lists valid options for each field.
API index: for gRPC libraries api_shortname should match the subdomain of an API's hostName.
Reach out to go/github-automation if you have any questions.

Core: ServiceOptions.setCredentials fails with ClassCastException when using UserCredentials

On line

java-core/google-cloud-core/src/main/java/com/google/cloud/ServiceOptions.java

Line 221 in 6aa4476

 this.quotaProjectId = ((ServiceAccountCredentials) credentials).getQuotaProjectId(); 

the check is instanceof QuotaProjectIdProvider but the cast is to ServiceAccountCredentials therefore you get

class com.google.auth.oauth2.UserCredentials cannot be cast to class com.google.auth.oauth2.ServiceAccountCredentials (com.google.auth.oauth2.UserCredentials and com.google.auth.oauth2.ServiceAccountCredentials are in unnamed module of loader 'app')

The cast needs to be changed. google-cloud-core version 1.92.0

Clean up tests for IAM Condition Support

Remaining clean up from PR #110
From: @netdpb
File: google-cloud-core/src/test/java/com/google/cloud/PolicyV3Test.java

You're calling a static method on an instance, and throwing the instance away. That's not what you want to do. (If the project used Error-Prone, this would have generated a warning.)

It looks like you want a variant of BINDINGS_NO_CONDITIONS where the viewer role has a condition. What about this?

ImmutableList.of(
    BINDINGS_NO_CONDITIONS.get(0).toBuilder().setCondition(...).build(),
    BINDINGS_NO_CONDITIONS.get(1))
Although it might be nice to have a constant for each element of BINDINGS_NO_CONDITIONS (VIEWER and EDITOR)?

Warning: a recent release failed

The release PR #206 is still in a pending state after several hours

docs: ServiceOptions should document exception for setting credentials twice

ServiceOptions.Builder#setCredentials throws a preconditions error if credentials are already set. It should be documented.

This module provides common functionality required by service-specific modules of this library.

In the readme "This module provides common functionality required by service-specific modules of this library." seems circular. I can;t quite figure out fro the readme what this repo is for. Can this be rewritten to be more specific and descriptive?

Warning: a recent release failed

The following release PRs may have failed:

#346

Support get CREDENTIAL_ENV_NAME from system properties

Right now, when we get the credential filename, the only way is from System.getenv(CREDENTIAL_ENV_NAME).

However, environment variable is immutable at runtime and hard to customized programmatically.

Thus, I suggest we also try to get the CREDENTIAL_ENV_NAME from system properties as we do for PROJECT_ENV_NAME.

Your .repo-metadata.json file has a problem 🤒

You have a problem with your .repo-metadata.json file:

Result of scan 📈:

must have required property 'client_documentation' in .repo-metadata.json

☝️ Once you address these problems, you can close this issue.

Need help?

Schema definition: lists valid options for each field.
API index: for gRPC libraries api_shortname should match the subdomain of an API's hostName.
Reach out to go/github-automation if you have any questions.

Native Image build is failing when the Category annotation is used in tests

Tests in the java-spanner projects used the Category annotation, which results in the following error when the native image is being built:

org.junit.platform.engine.TestTag was unintentionally initialized at build time. org.junit.platform.engine.TestTag caused initialization of this class with the following trace: 
	at org.junit.platform.engine.TestTag.<clinit>(TestTag.java:54)
  at org.junit.platform.engine.TestTag.<clinit>(TestTag.java:54)
	at org.junit.vintage.engine.descriptor.VintageTestDescriptor$$Lambda$656/0x00000007c23d8440.apply(Unknown Source)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
  ...
at org.junit.platform.engine.TestDescriptor.lambda$accept$0(TestDescriptor.java:251)
	at org.junit.platform.engine.TestDescriptor$$Lambda$639/0x00000007c23d4040.accept(Unknown Source)
	at java.lang.Iterable.forEach(Iterable.java:75)
	at org.junit.platform.engine.TestDescriptor.accept(TestDescriptor.java:251)
	at org.junit.platform.launcher.TestPlan.lambda$from$1(TestPlan.java:95)
	at org.junit.platform.launcher.TestPlan$$Lambda$649/0x00000007c23d6840.accept(Unknown Source)
	at java.util.LinkedHashMap$LinkedValues.forEach(LinkedHashMap.java:608)
	at java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1085)
	at org.junit.platform.launcher.TestPlan.from(TestPlan.java:95)
	at org.junit.platform.launcher.core.InternalTestPlan.from(InternalTestPlan.java:32)
	at org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:78)
	at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.discover(DefaultLauncherSession.java:81)
	at org.junit.platform.launcher.core.SessionPerRequestLauncher.discover(SessionPerRequestLauncher.java:46)
	at org.graalvm.junit.platform.JUnitPlatformFeature.discoverTestsAndRegisterTestClassesForReflection(JUnitPlatformFeature.java:135)

It looks like the failure occurs when VintageTestDescriptor calls the TestTag object which is not recognized by the native image builder.

Reproducer: https://github.com/mpeddada1/junit-experiment

Native image tests failing after upgrading google-cloud-shared-dependencies

The native image tests are breaking with the following error:

Error: Class initialization of io.grpc.netty.shaded.io.netty.internal.tcnative.AsyncSSLPrivateKeyMethod failed. Use the option --initialize-at-run-time=io.grpc.netty.shaded.io.netty.internal.tcnative.AsyncSSLPrivateKeyMethod to explicitly request delayed initialization of this class.

An example PR that is failing.

	/**
	* Creates a Timestamp instance from the given string. String is in the RFC 3339 format without
	* the timezone offset (always ends in "Z").
	*/
	public static Timestamp parseTimestamp(String timestamp) {

	import io.grpc.internal.SharedResourceHolder;
	import io.grpc.internal.SharedResourceHolder.Resource;

googleapis / java-core Goto Github PK

java-core's Introduction

Google Cloud Java Client -- Core

Java Versions

Contributing

Versioning

License

CI Status

java-core's People

Contributors

Stargazers

Watchers

Forkers

java-core's Issues

Repository problems

Open

Ignored or Blocked

Detected dependencies

Configuration

Problem

Environment details

Steps to reproduce

Code example

Stack trace

External references such as API reference guides

Any additional information below

Environment details

Steps to reproduce

Code example

Stack trace

External references such as API reference guides

Need help?

Need help?

Recommend Projects

Recommend Topics

Recommend Org

Jobs