Cloud Storage tests fail frequently on CircleCI about ruby-docs-samples

[beccasaurus]

Additional failing build. Spec needs fix.

https://travis-ci.org/GoogleCloudPlatform/ruby-docs-samples/jobs/163856203

[beccasaurus]

Thought: each test suite can use bucket [Project ID]-[Name of test suite]-[timestamp], eg. "TestProject-BigQuery-20160115123059.0462"

With our current Travis configuration...

• Multiple runs of all test suites may happen at the same time (multiple builds of ruby-docs-samples in parallel, eg. multiple pull requests)
• Multiple runs of test suites may happen at the same time within one overall build (as part of our current Travis matrix setup)

Vote: delete buckets after :all to not leave any buckets behind after each test suite run. Find or create in before each example block.

[frankyn]

Does this mean we will have overhead of creating a bucket for each test? It shouldn't be too bad.

[beccasaurus]

There are some specs that create a clean bucket context before each spec, eg. buckets_spec.rb

I think I'd like to update the specs to look like...

before do
  @test_bucket = create_test_bucket "BigQuery"
end

it "test that needs a second bucket" do
  @second_bucket = create_test_bucket "BigQuery second bucket"
end

Where #create_test_bucket is a required helper and the helper registers an after hook:

# */spec_helpers/storage_helper.rb
require "spec_helpers/project_helper.rb"

def test_bucket_names
  @test_bucket_names ||= []
end

def create_test_bucket bucket_suffix
  bucket_suffix = bucket_suffix.downcase.gsub /[^\w+]/, "_"
  bucket_name = "#{default_project_id}-#{bucket_suffic}-#{Time.now.to_f}"
  test_bucket_names << bucket_name
  Google::Cloud.new.storage.create_bucket bucket_name
end

def delete_test_buckets!
  test_bucket_names.each do |bucket_name|
    bucket = Google::Cloud.new.storage.bucket bucket_name
    bucket.delete if bucket
  end
end

RSpec.configure do |config|
  config.after do
    delete_test_buckets!
  end
end

Or something.

Some of the other docs samples repos generate buckets for their test runs using a timestamp which is good for isolation. Pretty simple.

[frankyn]

Gotcha, I like your idea for using RSpec after hooks 👍
Is it possible for concurrent tests to create the same bucket name? Time.now only goes up to seconds.

[beccasaurus]

Time in Ruby includes nanoseconds (since Epoch)

• Time#to_f
• Time#nsec

[frankyn]

Got it, I should have double checked that.

Thanks!

[beccasaurus]

I have not proven that this is the reason for Storage samples, so I am renaming to focus on fixing all flakiness in Storage sample tests.

[beccasaurus]

Upgrading all flaky test failures to p0

Example of Storage failures:

• Build 266
• Build 263

[frankyn]

Fixed in #211