CI hasn't passed in months about ruby-docs-samples

[broady]

/cc @frankyn @bshaffer

[frankyn]

I'll review errors after my next meeting. If you have time @bshaffer, I'd appreciate your help!

[frankyn]

@broady I started trying to address the issues in PR#211, and haven't completed this task. I'll see if I can green the tests. I'd like to introduce kokoro into the repo, but I haven't prioritized that. I'll see how far I get with fixing tests before I venture down that path.

If you have tips on how to best deal with this state I'd appreciate your input. Thank you!

[broady]

sounds good. i have a lot of things learned from deflaking the Go tests that I should share. a few things:

• retries within the tests
• using a project pool for proper resource segregation (I can help set this up)
• aggressive cleanup, aided by the project pool
• deterministic resource names, aided by the project pool

but it looks like there's a bunch of low hanging fruit for these ones. for example, it looks like the spanner db's aren't being cleaned up.

[frankyn]

Thanks Chris,

• Updated Spanner tests to be deterministic
• Added waiting before asserting an expectation on a logging.
• Updated Gemfile.lock files (need to do this automatically in the future as this is done locally and using a PR).
• Added additional Ruby versions to test on.
• Fixed a few small errors in a few other samples that made it into the repo.

I'm waiting for tests to finish to get an idea of where CircleCI is now. If you could help with proper resource segregation that would be awesome!

[frankyn]

Added retries to the following sample tests:
Storage
Pubsub
Datastore
Logging

Next item will be to add kokoro support unless we should introduce project resource pools first? @broady

[frankyn]

Status update: Master passed. Pubsub passed after retries kicked in when testing Ruby 2.4.2. Flaky test is pulls a message.

[broady]

Nice! Let's leave it at that for now.

[frankyn]

Sg!

[frankyn]

@broady, I recalled that AppEngine e2e tests are not being tested at the moment. The tests currently timeout CircleCI before completing. Reviewing in a follow-up PR.

[frankyn]

Ran e2e tests: https://circleci.com/gh/GoogleCloudPlatform/ruby-docs-samples/1225

I think this is a good reason to move over to kokoro as the e2e tests timeout CircleCI. An alternative is to split tests into multiple CircleCI nodes.

[broady]

Do the e2e tests run in parallel?

[frankyn]

No, they are not set up that way.

…

On Dec 12, 2017 9:14 AM, "Chris Broadfoot" ***@***.***> wrote: Do the e2e tests run in parallel? — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#218 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB0bwNX6l_8ClopAJKkPRjPlbASHIqFtks5s_rR_gaJpZM4Q4dSC> .

[broady]

@tbpg, could you provide an update here, or close if this is resolved? thanks!

[frankyn]

@broady progress is being tracked here: #251