elastic / connectors-ruby

Official Connector Clients for Elastic Elasticsearch, Enterprise Search, App Search and Workplace Search

Elastic Ruby connectors

The home of Elastic connector service and native connectors in Ruby language. This repository contains the framework for customizing Elastic native connectors, or writing your own connectors for advanced use cases.

Any connector implementation in this repository is only for reference, for supported versions please see connectors-python.

The connector will be operated by an administrative user from within Kibana.

Note: The connector framework is a tech preview feature. Tech preview features are subject to change and are not covered by the support SLA of general release (GA) features. Elastic plans to promote this feature to GA in a future release.

Before getting started, review important information about this feature:

• Terminology
• Getting help and providing feedback
• Understand the connector protocol

Build, deploy, and operate your own connector:

• Building/Deploying a connector
• Operating a connector

How to publish your connector:

• Contribute to the repository

Other guides

• Code of Conduct
• Getting Support
• Releasing
• Developer guide
• Security Policy
• Elastic-internal guide
• Connector Protocol
• Configuration
• Terminology
• Contributing guide

connectors-ruby's Issues

SharePoint security methodology is wrong - any plans to fix?

Just reviewing the SharePoint online connector security model and there are a number of issues, particularly around documents restricted to groups.

Are there any plans to redo the connector security component?

[SharePoint] Extend Metadata Indexing to custom one

Problem Description

While connecting an Elastic Cloud Instance to a Sharepoint Online using Workplace search, we can only get defaut metadata and not the custom attached to the hosted documents.

Proposed Solution

As it's a pretty standard usage as many organization have their own metadata labels to add context to their documents, Sharepoint Online connector should be able to access and index the custom metada.

Alternatives

A workaround is possible by adapting the logic in the sharepoint online connector package but

1. customer will have to modify the connector either himself or through PS
2. deploy this modified version into a dedicated cloud VPS instance to keep the solution over the cloud which is against a move cloud full SaaS strategy
3. No support from us until the custom code has been merged to the main branch and accepted.

Additional Context

This affect our ESS product

Unable to validate the web server

Hello Team,
I was able to build and run the custom connector using docker on the Ubuntu server. But unable to validate it.

Please find the below images for reference.

This is another test (ruby)

Bug Description

Sean is testing the ruby connectors now

To Reproduce

Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior

Screenshots

Environment

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Additional context

[SharePoint] Only traverse groups that user has access to and handle GraphAPI request errors

Bug Description

As part of an enhancement to sync private Teams sites without having to wait for a propagation delay in Microsoft, there is logic which loops through sites that are associated with groups returned by the Graph API "groups" endpoint.

The "groups" endpoint lists all the groups within the organization, including those that the connecting user may not have access to or that there might not be associated Sharepoint sites created. The end result as the sync is performed is that 404 and 403 issues are encountered and the sync process does not continue.

To Reproduce

Steps to reproduce the behavior:

1. Create groups that the connecting Azure service account does not have access to (referring to the account selected in Step 3 of the setup
2. Follow documentation to setup Sharepoint Online connector package
3. Perform sync
4. Observe that Sharepoint documents are not synced

Expected behavior

Sharepoint connector should not try to request information from groups it is not a direct/indirect member of. Sync should continue when encountering unsuccessful responses (e.g. 403, 404) and log this information.

Environment

• Tested on ESS 8.6.1
• Connector package installed on Ubuntu 20.04.5 LTS

Additional context

This is an example 404 error from the diagnostics bundle:

"createdAt": "2023-02-16T17:03:30Z",

          "status": "error",

          "fatalException": {

            "friendly_message": "Source returned an unexpected error during synchronization.",

            "stack_trace": "/home/username/enterprise-search-8.2.0/lib/war/gems/gems/connectors_sdk-8.2.0.0/lib/connectors_sdk/office365/custom_client.rb:316:in `raise_any_errors\u0027: got a 404 from https://graph.microsoft.com/v1.0/groups/01973b3f-dc6b-415b-b0ca-8f5e3c02c63c/sites/root with query {:$select\u003d\u003e\"id\"}\n\tfrom /home/username/enterprise-search-8.2.0/lib/war/gems/gems/connectors_sdk-8.2.0.0/lib/connectors_sdk/office365/custom_client.rb:331:in `request\u0027\n\tfrom /home/username/enterprise-search-8.2.0/lib/war/gems/gems/connectors_sdk-8.2.0.0/lib/connectors_sdk/office365/custom_client.rb:326:in `request_json\u0027\n\tfrom /home/username/enterprise-search-8.2.0/lib/war/gems/gems/connectors_sdk-8.2.0.0/lib/connectors_sdk/office365/custom_client.rb:322:in `request_endpoint\u0027\n\tfrom /home/username/enterprise-search-8.2.0/lib/war/gems/gems/connectors_sdk-8.2.0.0/lib/connectors_sdk/office365/custom_client.rb:95:in `group_root_site\u0027\n\tfrom /home/username/enterprise-search-8.2.0/lib/war/gems/gems/connectors_sdk-8.2.0.0/lib/connectors_sdk/office365/custom_client.rb:236:in `block in recent_share_point_group_sites\u0027\n\tfrom org/jruby/RubyArray.java:2642:in `map\u0027\n\tfrom /home/username/enterprise-search-8.2.0/lib/war/gems/gems/connectors_sdk-8.2.0.0/lib/connectors_sdk/office365/custom_client.rb:236:in `recent_share_point_group_sites\u0027\n\tfrom /home/username/enterprise-search-8.2.0/lib/war/gems/gems/connectors_sdk-8.2.0.0/lib/connectors_sdk/office365/custom_client.rb:78:in `share_point_drives\u0027\n\tfrom /home/username/enterprise-search-8.2.0/lib/war/gems/gems/connectors_sdk-8.2.0.0/lib/connectors_sdk/share_point/extractor.rb:27:in `drives\u0027\n\tfrom /home/username/enterprise-search-8.2.0/lib/war/gems/gems/connectors_sdk-8.2.0.0/lib/connectors_sdk/office365/extractor.rb:120:in `drives_to_index\u0027\n\tfrom /home/username/enterprise-search-8.2.0/lib/war/gems/gems/connectors_sdk-8.2.0.0/lib/connectors_sdk/office365/extractor.rb:75:in `retrieve_latest_cursors\u0027\n\tfrom /home/username/enterprise-search-8.2.0/lib/war/connectors/lib/connectors/work/purge.class:60:in `complete_run!\u0027\n\tfrom /home/username/enterprise-search-8.2.0/lib/war/connectors/lib/connectors/work/purge.class:35:in `block in run\u0027\n\tfrom /home/username/enterprise-search-8.2.0/lib/war/connectors/lib/connectors/work/abstract_extractor_work.class:100:in `run_with_suspension\u0027\n\tfrom /home/username/enterprise-search-8.2.0/lib/war/connectors/lib/connectors/work/purge.class:7:in `run\u0027\n\tfrom /home/username/enterprise-search-8.2.0/lib/war/connectors/lib/connectors/work/abstract_extractor_work.class:138:in `execute\u0027\n\tfrom /home/username/enterprise-search-8.2.0/lib/war/connectors/lib/connectors/workers/extract_worker.class:7:in `block in run\u0027\n\tfrom /home/username/enterprise-search-8.2.0/lib/war/gems/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/executor/safe_task_executor.rb:24:in `block in execute\u0027\n\tfrom /home/username/enterprise-search-8.2.0/lib/war/gems/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/executor/safe_task_executor.rb:19:in `execute\u0027\n\tfrom /home/username/enterprise-search-8.2.0/lib/war/connectors/config/initializers/concurrent.class:18:in `block in realize\u0027\n\tfrom /home/username/enterprise-search-8.2.0/lib/war/gems/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/executor/java_executor_service.rb:79:in `run\u0027\n",

            "id": "63ee62976786a6bbb45b0177",

            "message": "got a 404 from https://graph.microsoft.com/v1.0/groups/01973b3f-dc6b-415b-b0ca-8f5e3c02c63c/sites/root with query {:$select\u003d\u003e\"id\"}",

            "class": "ConnectorsSdk::Office365::CustomClient::ClientError"

          },

          "errorReason": "client_error",

          "completedAt": "2023-02-16T17:06:31Z",

          "durationSeconds": 181.0,

          "documentErrorCount": 0,

          "documentErrors": []

make api_key does not seem to be working on Windows 11

Bug Description

I tried following the install steps, ran make install then make api_key

Here was the output of make api_key

Its not really clear from the steps "Consumers will need to use the api_key string as the password in the basic Authorization header."

Am I meant to be seeing an api_key somewhere?

C:\Users\sysadmin\Downloads\connectors-ruby-8.3\connectors-ruby-8.3>make api_key

C:\Users\sysadmin\Downloads\connectors-ruby-8.3\connectors-ruby-8.3>rem

C:\Users\sysadmin\Downloads\connectors-ruby-8.3\connectors-ruby-8.3>rem Makefile for Windows

C:\Users\sysadmin\Downloads\connectors-ruby-8.3\connectors-ruby-8.3>rem

C:\Users\sysadmin\Downloads\connectors-ruby-8.3\connectors-ruby-8.3>rem This batch script makes the assumption that your rbenv environment is installed.

C:\Users\sysadmin\Downloads\connectors-ruby-8.3\connectors-ruby-8.3>rem A fully automated install script is provided in win32\install.bat

C:\Users\sysadmin\Downloads\connectors-ruby-8.3\connectors-ruby-8.3>rem
"Ruby 2.6.9 - Bundler 2.3.10"
"Install gem dependencies..."
Successfully installed bundler-2.3.10
Parsing documentation for bundler-2.3.10
Done installing documentation for bundler after 0 seconds
1 gem installed
[DEPRECATED] The `--with` flag is deprecated because it relies on being remembered across bundler invocations, which bundler will no longer do in future versions. Instead please use `bundle config set --local with 'test'`, and stop using this flag
Using rake 13.0.6
Using concurrent-ruby 1.1.9
Using i18n 1.10.0
Using minitest 5.15.0
Using thread_safe 0.3.6
Using tzinfo 1.2.9
Using activesupport 5.2.6
Using public_suffix 4.0.6
Using addressable 2.8.0
Using ast 2.4.2
Using attr_extras 6.2.5
Using bson 4.2.2
Using bundler 2.3.10
Using coderay 1.1.3
Using deep_merge 1.2.2
Using dry-core 0.7.1
Using dry-configurable 0.13.0
Using dry-container 0.9.0
Using dry-initializer 3.0.4
Using dry-logic 1.2.0
Using dry-inflector 0.2.1
Using dry-types 1.5.1
Using dry-schema 1.8.0
Using dry-validation 1.7.0
Using config 4.0.0
Using rexml 3.2.5
Using crack 0.4.5
Using debase-ruby_core_source 0.10.14
Using debase 0.2.5.beta2
Using diff-lcs 1.5.0
Using docile 1.4.0
Using faraday-em_http 1.0.0
Using faraday-em_synchrony 1.0.0
Using faraday-excon 1.1.0
Using faraday-httpclient 1.0.1
Using multipart-post 2.1.1
Using faraday-multipart 1.0.3
Using faraday-net_http 1.0.1
Using faraday-net_http_persistent 1.2.0
Using faraday-patron 1.0.0
Using faraday-rack 1.0.0
Using faraday-retry 1.0.3
Using ruby2_keywords 0.0.5
Using faraday 1.10.0
Using faraday_middleware 1.0.0
Using forwardable 1.3.2
Using hashdiff 1.0.1
Using hashie 5.0.0
Using httpclient 2.8.3
Using jwt 2.3.0
Using method_source 1.0.0
Using mime-types-data 3.2022.0105
Using mime-types 3.1
Using multi_json 1.15.0
Using mustermann 1.1.1
Using racc 1.6.0
Using nokogiri 1.13.4 (x86-mingw32)
Using parallel 1.21.0
Using parser 3.1.1.0
Using pry 0.14.1
Using pry-nav 1.0.0
Using slop 3.6.0
Using pry-remote 0.1.8
Using rack 2.2.3
Using rack-protection 2.1.0
Using rack-test 1.1.0
Using rainbow 3.1.1
Using regexp_parser 2.2.1
Using rspec-support 3.10.3
Using rspec-expectations 3.10.2
Using rspec-collection_matchers 1.2.0
Using rspec-core 3.10.2
Using rspec-mocks 3.10.3
Using rspec_junit_formatter 0.5.1
Using rubocop-ast 1.16.0
Using ruby-progressbar 1.11.0
Using unicode-display_width 2.1.0
Using rubocop 1.18.4
Using rubocop-performance 1.11.5
Using ruby-debug-ide 0.7.3
Using signet 0.16.0
Using simplecov-html 0.12.3
Using simplecov_json_formatter 0.1.4
Using simplecov 0.21.2
Using simplecov-material 1.0.0
Using tilt 2.0.10
Using sinatra 2.1.0
Using sinatra-contrib 2.1.0
Using timecop 0.9.4
Using tzinfo-data 1.2022.1
Using webmock 3.14.0
Bundle complete! 33 Gemfile dependencies, 91 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
"Running tests..."
        1 file(s) copied.
Parsing C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/lib/connectors_app/../../config/connectors.yml configuration file.
I, [2022-07-21T09:53:54.638104 #19152]  INFO -- : Watching after jobs {•̃_•̃}

Randomized with seed 8478

ConnectorsSdk::SharePoint::Extractor
  #yield_document_changes
    filters out unused drive types
      when personal
      when business
    document-level permissions disabled
      has sites prefix in path
      does not set permissions
      with cursor
        yields document
      documents fail
W, [2022-07-21T09:53:54.915689 #19152]  WARN -- : ContentSource[62d89592b256b94ad0609fd0, sharepoint_online]: Encountered error during extraction of 'Office365 change: 8d380547c0b366c5ac2b269b2af7e297 (/33ee7824410cda068105dc2500fba268.docx)': RuntimeError: error {:message_id => 62d89592b256b94ad0609fd1}
        bypasses single document failures
      incremental sync
        yields document
    document-level permissions enabled
      sets the document permissions if available
      sets no deny permissions
      sets no permissions if none available
  #yield_deleted_ids
    with removed item
      yields the deleted item id
    with no item removed
      yields nothing
  #drive_ids
    when drive ids are all
      should grab all the drive ids
    when drive ids are a specific set
      should return those drive ids

ConnectorsAsync::Job
  #update_cursors
    updates cursors
  #has_cursors?
    when cursors were set
      returns true
    when no cursors were set
      returns false
  #fail
    changes job status to failed
    stores an error
  #initialize
    sets CREATED status
    sets the same id that was passed into the constructor
  #pop_batch
    when no documents were stored
      returns an empty array
    when a threading error occurs
Attempt to access an empty queue happened, when the queue was not supposed to be empty
      does not throw the ThreadingError
Attempt to access an empty queue happened, when the queue was not supposed to be empty
      returns the results that got out before the error
    when less than up_to documents were stored
      returns all stored docs
  #update_status
    when passed status is invalid
      raises an error
    when job has finished
      raises an error
    when job hasn't finished
      allows to update status
  #store
    stores an item that can later be retrieved
  #is_failed?
    when job hasn't failed before
      returns false
    when job failed
      returns true

ConnectorsSdk::Base::Connector
  .extract
    yields documents
  .deleted
    with invalid access token
      raise InvalidTokenError
    with valid access token
      returns deleted ids
  .download
    calls extractor.download method with same params
    when extractor raises an error
C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/spec/connectors_sdk/base/connector_spec.rb:106: warning: constant ::TimeoutError is deprecated
      does not suppress errors from extractor
  .permissions
    with valid access token
      returns permissions
    with invalid access token
      raise InvalidTokenError
  .compare_secrets
    when other_secret is missing
      raises ClientError
    when params is empty
      raises ClientError
    when secret is missing
      raises ClientError
  .source_status
    remote source is up
      returns OK status
    remote source is down
      returns FAILURE status

ConnectorsAsync::JobWatcher
  when a finished job can be cleaned up
I, [2022-07-21T09:53:55.184681 #19152]  INFO -- : Watching after jobs {•̃_•̃}
    attempts to remove the job without failing it
  when something raises an exception in the watcher loop
I, [2022-07-21T09:53:55.188058 #19152]  INFO -- : Watching after jobs {•̃_•̃}
E, [2022-07-21T09:53:55.189026 #19152] ERROR -- : Traceback (most recent call last):
        77: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/bin/rspec:23:in `<main>'
        76: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/bin/rspec:23:in `load'
        75: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/exe/rspec:4:in `<top (required)>'
        74: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/runner.rb:45:in `invoke'
        73: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/runner.rb:71:in `run'
        72: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/runner.rb:89:in `run'
        71: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/runner.rb:115:in `run_specs'
        70: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/reporter.rb:74:in `report'
        69: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/runner.rb:116:in `block in run_specs'
        68: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/configuration.rb:2067:in `with_suite_hooks'
        67: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/runner.rb:121:in `block (2 levels) in run_specs'
        66: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/runner.rb:121:in `map'
        65: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/runner.rb:121:in `block (3 levels) in run_specs'
        64: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example_group.rb:607:in `run'
        63: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example_group.rb:607:in `map'
        62: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example_group.rb:607:in `block in run'
        61: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example_group.rb:606:in `run'
        60: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example_group.rb:640:in `run_examples'
        59: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example_group.rb:640:in `map'
        58: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example_group.rb:644:in `block in run_examples'
        57: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:259:in `run'
        56: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:508:in `with_around_and_singleton_context_hooks'
        55: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:465:in `with_around_example_hooks'
        54: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/hooks.rb:486:in `run'
        53: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/hooks.rb:629:in `run_around_example_hooks_for'
        52: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:350:in `call'
        51: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/hooks.rb:628:in `block (2 levels) in run_around_example_hooks_for'
        50: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/hooks.rb:390:in `execute_with'
        49: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:455:in `instance_exec'
        48: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:455:in `instance_exec'
        47: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/webmock-3.14.0/lib/webmock/rspec.rb:37:in `block (2 levels) in <top (required)>'
        46: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:350:in `call'
        45: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/hooks.rb:626:in `block in run_around_example_hooks_for'
        44: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/hooks.rb:486:in `block in run'
        43: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:465:in `block in with_around_example_hooks'
        42: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:508:in `block in with_around_and_singleton_context_hooks'
        41: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:262:in `block in run'
        40: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:262:in `instance_exec'
        39: from C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/spec/connectors_async/job_watcher_spec.rb:57:in `block (3 levels) in <top (required)>'
        38: from C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/lib/connectors_async/job_watcher.rb:30:in `watch'
        37: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-mocks-3.10.3/lib/rspec/mocks/method_double.rb:64:in `block (2 levels) in define_proxy_method'
        36: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-mocks-3.10.3/lib/rspec/mocks/method_double.rb:78:in `proxy_method_invoked'
        35: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-mocks-3.10.3/lib/rspec/mocks/proxy.rb:361:in `message_received'
        34: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-mocks-3.10.3/lib/rspec/mocks/proxy.rb:214:in `message_received'
        33: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-mocks-3.10.3/lib/rspec/mocks/message_expectation.rb:428:in `invoke'
        32: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-mocks-3.10.3/lib/rspec/mocks/message_expectation.rb:573:in `invoke_incrementing_actual_calls_by'
        31: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-mocks-3.10.3/lib/rspec/mocks/message_expectation.rb:694:in `call'
        30: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-mocks-
...snip...
W, [2022-07-21T09:53:56.668408 #19152]  WARN -- : ContentSource[62d89594b256b94ad0609ff3, share_point]: Failed to connect in with_auth_tokens_and_retry. Reason: RuntimeError: fail {:message_id => 62d89594b256b94ad0609ff2}
W, [2022-07-21T09:53:56.668798 #19152]  WARN -- : ContentSource[62d89594b256b94ad0609ff3, share_point]: Retries: 1/3, trying again.
W, [2022-07-21T09:53:56.669109 #19152]  WARN -- : ContentSource[62d89594b256b94ad0609ff3, share_point]: Failed to connect in with_auth_tokens_and_retry. Reason: RuntimeError: fail {:message_id => 62d89594b256b94ad0609ff4}
W, [2022-07-21T09:53:56.669790 #19152]  WARN -- : ContentSource[62d89594b256b94ad0609ff3, share_point]: Retries: 2/3, trying again.
W, [2022-07-21T09:53:56.670100 #19152]  WARN -- : ContentSource[62d89594b256b94ad0609ff3, share_point]: Failed to connect in with_auth_tokens_and_retry Reason: RuntimeError: fail {:message_id => 62d89594b256b94ad0609ff5}
W, [2022-07-21T09:53:56.670719 #19152]  WARN -- : ContentSource[62d89594b256b94ad0609ff3, share_point]: Retries: 3/3, giving up.
E, [2022-07-21T09:53:56.671705 #19152] ERROR -- : Traceback (most recent call last):
        74: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/bin/rspec:23:in `<main>'
        73: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/bin/rspec:23:in `load'
        72: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/exe/rspec:4:in `<top (required)>'
        71: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/runner.rb:45:in `invoke'
        70: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/runner.rb:71:in `run'
        69: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/runner.rb:89:in `run'
        68: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/runner.rb:115:in `run_specs'
        67: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/reporter.rb:74:in `report'
        66: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/runner.rb:116:in `block in run_specs'
        65: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/configuration.rb:2067:in `with_suite_hooks'
        64: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/runner.rb:121:in `block (2 levels) in run_specs'
        63: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/runner.rb:121:in `map'
        62: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/runner.rb:121:in `block (3 levels) in run_specs'
        61: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example_group.rb:607:in `run'
        60: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example_group.rb:607:in `map'
        59: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example_group.rb:607:in `block in run'
        58: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example_group.rb:607:in `run'
        57: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example_group.rb:607:in `map'
        56: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example_group.rb:607:in `block in run'
        55: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example_group.rb:607:in `run'
        54: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example_group.rb:607:in `map'
        53: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example_group.rb:607:in `block in run'
        52: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example_group.rb:606:in `run'
        51: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example_group.rb:640:in `run_examples'
        50: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example_group.rb:640:in `map'
        49: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example_group.rb:644:in `block in run_examples'
        48: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:259:in `run'
        47: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:508:in `with_around_and_singleton_context_hooks'
        46: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:465:in `with_around_example_hooks'
        45: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/hooks.rb:486:in `run'
        44: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/hooks.rb:629:in `run_around_example_hooks_for'
        43: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:350:in `call'
        42: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/hooks.rb:628:in `block (2 levels) in run_around_example_hooks_for'
        41: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/hooks.rb:390:in `execute_with'
        40: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:455:in `instance_exec'
        39: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:455:in `instance_exec'
        38: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/webmock-3.14.0/lib/webmock/rspec.rb:37:in `block (2 levels) in <top (required)>'
        37: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:350:in `call'
        36: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/hooks.rb:626:in `block in run_around_example_hooks_for'
        35: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/hooks.rb:486:in `block in run'
        34: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:465:in `block in with_around_example_hooks'
        33: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:508:in `block in with_around_and_singleton_context_hooks'
        32: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:262:in `block in run'
        31: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-core-3.10.2/lib/rspec/core/example.rb:262:in `instance_exec'
        30: from C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/spec/connectors_sdk/base/extractor_spec.rb:161:in `block (5 levels) in <top (required)>'
        29: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-expectations-3.10.2/lib/rspec/expectations/expectation_target.rb:101:in `to'
        28: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-expectations-3.10.2/lib/rspec/expectations/expectation_target.rb:65:in `to'
        27: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-expectations-3.10.2/lib/rspec/expectations/handler.rb:48:in `handle_matcher'
        26: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-expectations-3.10.2/lib/rspec/expectations/handler.rb:27:in `with_matcher'
        25: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-expectations-3.10.2/lib/rspec/expectations/handler.rb:51:in `block in handle_matcher'
        24: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-expectations-3.10.2/lib/rspec/matchers/built_in/raise_error.rb:59:in `matches?'
        23: from C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/spec/connectors_sdk/base/extractor_spec.rb:162:in `block (6 levels) in <top (required)>'
        22: from C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/lib/connectors_sdk/base/extractor.rb:124:in `document_changes'
        21: from C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/lib/stubs/connectors/stats.rb:14:in `measure'
        20: from C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/lib/connectors_sdk/base/extractor.rb:125:in `block in document_changes'
        19: from C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/lib/connectors_sdk/base/extractor.rb:81:in `with_auth_tokens_and_retry'
        18: from C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/lib/connectors_sdk/base/extractor.rb:216:in `convert_transient_server_errors'
        17: from C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/lib/connectors_sdk/base/extractor.rb:82:in `block in with_auth_tokens_and_retry'
        16: from C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/lib/connectors_sdk/base/extractor.rb:246:in `convert_rate_limit_errors'
        15: from C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/lib/connectors_sdk/base/extractor.rb:126:in `block (2 levels) in document_changes'
        14: from C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/lib/stubs/connectors/stats.rb:14:in `measure'
        13: from C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/lib/connectors_sdk/base/extractor.rb:135:in `block (3 levels) in document_changes'
        12: from C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/lib/connectors_sdk/base/extractor.rb:135:in `each'
        11: from C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/lib/connectors_sdk/base/extractor.rb:135:in `each'
        10: from C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/lib/connectors_sdk/base/extractor.rb:129:in `block (4 levels) in document_changes'
         9: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-mocks-3.10.3/lib/rspec/mocks/method_double.rb:64:in `block (2 levels) in define_proxy_method'
         8: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-mocks-3.10.3/lib/rspec/mocks/method_double.rb:78:in `proxy_method_invoked'
         7: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-mocks-3.10.3/lib/rspec/mocks/proxy.rb:361:in `message_received'
         6: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-mocks-3.10.3/lib/rspec/mocks/proxy.rb:217:in `message_received'
         5: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-mocks-3.10.3/lib/rspec/mocks/message_expectation.rb:428:in `invoke'
         4: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-mocks-3.10.3/lib/rspec/mocks/message_expectation.rb:573:in `invoke_incrementing_actual_calls_by'
         3: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-mocks-3.10.3/lib/rspec/mocks/message_expectation.rb:741:in `call'
         2: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-mocks-3.10.3/lib/rspec/mocks/message_expectation.rb:101:in `block in and_call_original'
         1: from C:/Users/sysadmin/.rbenv-win/versions/2.6.9/lib/ruby/gems/2.6.0/gems/rspec-mocks-3.10.3/lib/rspec/mocks/message_expectation.rb:101:in `call'
C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/spec/connectors_sdk/base/extractor_spec.rb:135:in `yield_document_changes': fail (RuntimeError)

        retries multiple times and finally raises the error
    when a rate limit error is raised
E, [2022-07-21T09:53:56.682252 #19152] ERROR -- : ContentSource[62d89594b256b94ad0609ff6, share_point]: Encountered a fall-through error during extraction of 'the only one': ConnectorsShared::ThrottlingError: ConnectorsShared::ThrottlingError {:message_id => 62d89594b256b94ad0609ff7}
      does not retry and raises the error
  #permissions
I, [2022-07-21T09:53:56.684075 #19152]  INFO -- : ContentSource[62d89594b256b94ad0609ff8, share_point]: Extracted 2 permissions for source user id
    yields permissions

ConnectorsShared::ExceptionTracking
  can log an exception

ConnectorsSdk::GitLab::Adapter
  #to_es_document
    correctly produced the Enterprise Search ID
    fills in all the other data
    with permissions
      fills in permissions

ConnectorsAsync::JobStore
  #delete_job!
    when id of existing job is passed
I, [2022-07-21T09:53:56.698026 #19152]  INFO -- : Watching after jobs {•̃_•̃}
      removes this job
    when id of non-existing job is passed
I, [2022-07-21T09:53:56.699737 #19152]  INFO -- : Watching after jobs {•̃_•̃}
      raises an error
  #fetch_all
I, [2022-07-21T09:53:56.701879 #19152]  INFO -- : Watching after jobs {•̃_•̃}
    returns all stored jobs
  #fetch_job
    when id of non-existing job is passed
I, [2022-07-21T09:53:56.706246 #19152]  INFO -- : Watching after jobs {•̃_•̃}
      raises an error
    when a job was created before
I, [2022-07-21T09:53:56.710247 #19152]  INFO -- : Watching after jobs {•̃_•̃}
      returns a job
  #create_job
I, [2022-07-21T09:53:56.711975 #19152]  INFO -- : Watching after jobs {•̃_•̃}
    returns a job with generated id
I, [2022-07-21T09:53:56.712540 #19152]  INFO -- : Watching after jobs {•̃_•̃}
    returns a job with unique id

ConnectorsSdk::ConfluenceCloud::Extractor
  has the right client middleware
  can initialize a client
  attachment downloading
    downloads from the right path
  with permissions
I, [2022-07-21T09:53:56.768674 #19152]  INFO -- : ContentSource[62d89594b256b94ad0609ff9, confluence_cloud]: Extracted 2 permissions for source user abcd1234
    looks up the user
I, [2022-07-21T09:53:56.790931 #19152]  INFO -- : ContentSource[62d89594b256b94ad0609ffa, confluence_cloud]: Extracted 2 permissions for source user abcd1234
    looks up the user groups
    when user is suspended/deleted
      yields no permissions
    when user has two groups and access to space via one
      yields permissions
    when user has access via group only
      yields permissions
    when users and groups have no space access
      yields no permissions
    with space with anonymous access
      does not attach permissions
      makes a request to the spaces
      with page without restrictions
        does not attach permissions
      with page with restrictions
        attaches permissions
    when users and groups have space access
      yields the permissions
    when user has access via user only
      yields permissions
    with space without anonymous access
      makes a request to the spaces
      attaches permissions
      with child page without restrictions
        attaches permissions
      with page without restrictions
        attaches permissions
      with page with restrictions
        attaches permissions
      with attachment to a page with restrictions
        attaches permissions
      with attachment to a space
I, [2022-07-21T09:53:57.105459 #19152]  INFO -- : ContentSource[62d89595b256b94ad060a005, confluence_cloud]: Skipping ancestor restrictions as it is a space and should already be cached: [10] in [global_9]
        attaches permissions
      with attachment to a child page without restrictions
        attaches permissions
      with real life example
        attaches permissions
      with attachment to a page without restrictions
        attaches permissions

Repository
  NOTICE file
    exists
    is included in our Gemspecs
  LICENSE file
    exists
    is included in our Gemspecs
  License header
    prefixes all code files

ConnectorsSdk::Base::Factory
  let us register a connector under a name

ConnectorsSdk::Atlassian::CustomClient
  #download
    will follow a redirect
    will download contents from a url

ConnectorsShared::ExtractionUtils
  .node_descendant_text
    should replace break tags with spaces
    should raise an error unless given a node object
    with uncrate.com pages
      should have a reasonable performance

ConnectorsWebApp
  POST /oauth2/init
    with invalid request
      returns bad request
    with valid request
      returns authorization uri
  Catch all
    returns a 500 in JSON, on any error
  GET /
    returns the connectors metadata
  POST /download
    when valid parameters are passed
      returns the result of Connector.download
    when JSON.parse raises an error
      raises this error
    when Connector.download raises an error
      raises this error
  POST /oauth2/exchange
    with valid request
      returns tokens
    with invalid request
      returns bad request
  POST /status
    returns source status
  Authorization /
    returns a 401 on malformed Basic auth
    returns a 401 on unsupported auth scheme
    returns a 200 when Basic auth is OK
    returns a 401 when Basic auth is wrong
    returns a 401 when Basic auth misses
  POST /secrets/compare
    compares secrets
  POST /permissions
    returns deleted ids
  POST /deleted
    returns deleted ids
  POST /oauth2/refresh
    with valid request
      with expired refresh token
        returns 401
      with valid refresh token
        returns tokens
    with invalid request
      returns bad request

ConnectorsSdk::GitLab::Extractor
  #permissions
I, [2022-07-21T09:53:57.348909 #19152]  INFO -- : ContentSource[gitlab_source_1, gitlab]: Extracted 1 permissions for source user 11422639
    correctly sets permissions for external user
I, [2022-07-21T09:53:57.360957 #19152]  INFO -- : ContentSource[gitlab_source_1, gitlab]: Extracted 2 permissions for source user 1
    correctly sets permissions for internal user
  #document_changes
I, [2022-07-21T09:53:57.381163 #19152]  INFO -- : ContentSource[gitlab_source_1, gitlab]: Extracted 100 documents so far
    correctly produces one page of documents
    for incremental sync
I, [2022-07-21T09:53:57.400397 #19152]  INFO -- : ContentSource[gitlab_source_1, gitlab]: Extracted 100 documents so far
      uses the modified after date
    for multi-page results
      with permissions
        internal projects
          returns internal in permissions
        public projects
          returns empty permissions
        private projects
          returns actual users in permissions
      when next page
I, [2022-07-21T09:53:57.462115 #19152]  INFO -- : ContentSource[gitlab_source_1, gitlab]: Extracted 100 documents so far
I, [2022-07-21T09:53:57.468920 #19152]  INFO -- : ContentSource[gitlab_source_1, gitlab]: Extracted 200 documents so far
        uses the cursor link from parameters
      without permissions
        private projects
          returns nothing in permissions
  #deleted
    correctly gets non-existing ids

ConnectorsSdk::Confluence::CustomClient
  does not apply incorrect middleware for token auth
  #search
  applies correct middleware for token auth
  #content_by_id
  #spaces
  #content_search
  applies correct middleware for token auth
  does not apply incorrect middleware for basic auth
  inherits from Atlassian
  applies correct middleware for basic auth
  #content
    issues a content search
    prefers a next_value parameter
  error when non-200 response
    raises ServiceUnavailableError for 504
      behaves like failure
        raises
    raises ClientError otherwise
      behaves like failure
        raises
    raises ContentConvertibleError for 400
      behaves like failure
        raises

ConnectorsSdk::Base::CustomClient
  #delete
  #post
  #put
  #get
  retries
    only retries MAX_RETRIES times
    retries on timeout response
  ensuring auth is fresh
    does not require refresh logic
    auth refresh logic is provided
      will use refresh logic when supplied
  #request_with_throttling
    when rate limit is reached
W, [2022-07-21T09:54:00.820243 #19152]  WARN -- : Exceeded ConnectorsSdk::Base::CustomClient request limits. Going to sleep for 60 seconds
      raises ThrottlingError
    when request is successful
      returns 200

Pending: (Failures listed here are expected and do not affect your suite's status)

  1) ConnectorsShared::Middleware::RestrictHostnames when localhost URLs are allowed by hostname localhost ipv6 jenkins docker does not bind localhost to ipv6
     # No reason given
     # ./spec/connectors_shared/middleware/restrict_hostnames_spec.rb:82

  2) ConnectorsShared::Middleware::RestrictHostnames when localhost IPV6 URLs are allowed localhost domain jenkins docker does not bind localhost to ipv6
     # No reason given
     # ./spec/connectors_shared/middleware/restrict_hostnames_spec.rb:137


Top 10 slowest examples (4.02 seconds, 66.3% of total time):
  ConnectorsSdk::Base::CustomClient retries only retries MAX_RETRIES times
    1.6 seconds ./spec/connectors_sdk/base/custom_client_spec.rb:49
  ConnectorsSdk::Base::CustomClient#request_with_throttling when rate limit is reached raises ThrottlingError
    1.59 seconds ./spec/connectors_sdk/base/custom_client_spec.rb:94
  ConnectorsAsync::JobRunner#start_job when job indicates that no results were picked up for a bit throttles the run until the job no longer indicates it should be throttled
    0.11378 seconds ./spec/connectors_async/job_runner_spec.rb:191
  ConnectorsAsync::JobRunner#start_job when extractor completes immediately when extractor returns results stores docs in the job
    0.10759 seconds ./spec/connectors_async/job_runner_spec.rb:120
  ConnectorsAsync::JobRunner#start_job when extractor calls ConnectorsSdk::Base::Adapter.normalize_data does not raise an error
    0.10718 seconds ./spec/connectors_async/job_runner_spec.rb:170
  ConnectorsAsync::JobRunner#start_job when extractor completes immediately when extractor raises an error fails job
    0.10701 seconds ./spec/connectors_async/job_runner_spec.rb:144
  ConnectorsAsync::JobRunner#start_job when extractor completes immediately updates the job status throughout the run
    0.10667 seconds ./spec/connectors_async/job_runner_spec.rb:77
  ConnectorsAsync::JobRunner#start_job when job indicates that no results were picked up for a bit when job idled for too long fails a job
    0.10492 seconds ./spec/connectors_async/job_runner_spec.rb:209
  ConnectorsAsync::JobRunner#start_job when extractor completes immediately updates the cursors with cursors received from extractor
    0.10393 seconds ./spec/connectors_async/job_runner_spec.rb:91
  ConnectorsSdk::Office365::CustomClient retries on a 429 response
    0.07586 seconds ./spec/connectors_sdk/office365/custom_client_spec.rb:80

Top 10 slowest example groups:
  ConnectorsSdk::Base::CustomClient
    0.3281 seconds average (3.28 seconds / 10 examples) ./spec/connectors_sdk/base/custom_client_spec.rb:11
  ConnectorsAsync::JobRunner
    0.09623 seconds average (0.76986 seconds / 8 examples) ./spec/connectors_async/job_runner_spec.rb:13
  ConnectorsSdk::Office365::CustomClient
    0.02416 seconds average (0.19331 seconds / 8 examples) ./spec/connectors_sdk/office365/custom_client_spec.rb:13
  ConnectorsSdk::SharePoint::Extractor
    0.02017 seconds average (0.28235 seconds / 14 examples) ./spec/connectors_sdk/sharepoint/extractor_spec.rb:13
  ConnectorsSdk::ConfluenceCloud::Extractor
    0.01976 seconds average (0.49389 seconds / 25 examples) ./spec/connectors_sdk/confluence_cloud/extractor_spec.rb:15
  ConnectorsSdk::GitLab::Extractor
    0.01581 seconds average (0.15806 seconds / 10 examples) ./spec/connectors_sdk/gitlab/extractor_spec.rb:9
  ConnectorsSdk::Office365::Extractor
    0.0151 seconds average (0.09063 seconds / 6 examples) ./spec/connectors_sdk/office365/extractor_spec.rb:15
  ConnectorsSdk::Confluence::Extractor
    0.01185 seconds average (0.10663 seconds / 9 examples) ./spec/connectors_sdk/confluence/extractor_spec.rb:16
  ConnectorsSdk::GitLab::Connector
    0.01015 seconds average (0.03045 seconds / 3 examples) ./spec/connectors_sdk/gitlab/connector_spec.rb:6
  ConnectorsShared::ExtractionUtils
    0.00823 seconds average (0.0247 seconds / 3 examples) ./spec/connectors_shared/extraction_utils_spec.rb:14

Finished in 6.05 seconds (files took 2.46 seconds to load)
287 examples, 0 failures, 2 pending

Randomized with seed 8478

Coverage report generated for RSpec to C:/Users/sysadmin/Downloads/connectors-ruby-8.3/connectors-ruby-8.3/coverage. 1888 / 2114 LOC (89.31%) covered.

C:\Users\sysadmin\Downloads\connectors-ruby-8.3\connectors-ruby-8.3>

To Reproduce

Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior

Screenshots

Environment

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Additional context

TZInfo::DataSourceNotFound - No source of timezone data could be found.

Looks like we are missing tzinfo-data when running in Ubuntu Focal in Docker

cd lib/app; bundle exec rackup --host 0.0.0.0 config.ru
[2022-03-31 10:54:34] INFO  WEBrick 1.4.4
[2022-03-31 10:54:34] INFO  ruby 2.6.9 (2021-11-24) [x86_64-linux]
[2022-03-31 10:54:34] INFO  WEBrick::HTTPServer#start: pid=32 port=9292
2022-03-31 10:54:39 - TZInfo::DataSourceNotFound - No source of timezone data could be found.
Please refer to https://tzinfo.github.io/datasourcenotfound for help resolving this error.:
	/app/vendor/bundle/ruby/2.6.0/gems/tzinfo-1.2.9/lib/tzinfo/data_source.rb:182:in `rescue in create_default_data_source'
	/app/vendor/bundle/ruby/2.6.0/gems/tzinfo-1.2.9/lib/tzinfo/data_source.rb:179:in `create_default_data_source'
	/app/vendor/bundle/ruby/2.6.0/gems/tzinfo-1.2.9/lib/tzinfo/data_source.rb:40:in `block in get'
	/app/vendor/bundle/ruby/2.6.0/gems/tzinfo-1.2.9/lib/tzinfo/data_source.rb:39:in `synchronize'
	/app/vendor/bundle/ruby/2.6.0/gems/tzinfo-1.2.9/lib/tzinfo/data_source.rb:39:in `get'
	/app/vendor/bundle/ruby/2.6.0/gems/tzinfo-1.2.9/lib/tzinfo/timezone.rb:665:in `data_source'
	/app/vendor/bundle/ruby/2.6.0/gems/tzinfo-1.2.9/lib/tzinfo/timezone.rb:92:in `get'
	/app/vendor/bundle/ruby/2.6.0/gems/tzinfo-1.2.9/lib/tzinfo/timezone.rb:113:in `new'
	/app/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.6/lib/active_support/values/time_zone.rb:206:in `find_tzinfo'
	/app/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.6/lib/active_support/values/time_zone.rb:300:in `initialize'
	/app/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.6/lib/active_support/values/time_zone.rb:234:in `new'
	/app/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.6/lib/active_support/values/time_zone.rb:234:in `[]'
	/app/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.6/lib/active_support/values/time_zone.rb:215:in `new'
	/app/lib/app/app.rb:57:in `block in <class:ConnectorsWebApp>'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1675:in `call'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1675:in `block in compile!'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1061:in `block in process_route'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1059:in `catch'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1059:in `process_route'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1000:in `block in filter!'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:999:in `each'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:999:in `filter!'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1126:in `block in dispatch!'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `block in invoke'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `catch'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `invoke'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1124:in `dispatch!'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:939:in `block in call!'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `block in invoke'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `catch'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `invoke'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:939:in `call!'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:929:in `call'
	/app/vendor/bundle/ruby/2.6.0/gems/rack-protection-2.1.0/lib/rack/protection/xss_header.rb:18:in `call'
	/app/vendor/bundle/ruby/2.6.0/gems/rack-protection-2.1.0/lib/rack/protection/path_traversal.rb:16:in `call'
	/app/vendor/bundle/ruby/2.6.0/gems/rack-protection-2.1.0/lib/rack/protection/json_csrf.rb:26:in `call'
	/app/vendor/bundle/ruby/2.6.0/gems/rack-protection-2.1.0/lib/rack/protection/base.rb:50:in `call'
	/app/vendor/bundle/ruby/2.6.0/gems/rack-protection-2.1.0/lib/rack/protection/base.rb:50:in `call'
	/app/vendor/bundle/ruby/2.6.0/gems/rack-protection-2.1.0/lib/rack/protection/frame_options.rb:31:in `call'
	/app/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/null_logger.rb:11:in `call'
	/app/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/head.rb:12:in `call'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:216:in `call'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1991:in `call'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1542:in `block in call'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1769:in `synchronize'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1542:in `call'
	/app/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/tempfile_reaper.rb:15:in `call'
	/app/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/lint.rb:50:in `_call'
	/app/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/lint.rb:38:in `call'
	/app/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/show_exceptions.rb:23:in `call'
	/app/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/common_logger.rb:38:in `call'
	/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:253:in `call'
	/app/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/content_length.rb:17:in `call'
	/app/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/handler/webrick.rb:95:in `service'
	/root/.rbenv/versions/2.6.9/lib/ruby/2.6.0/webrick/httpserver.rb:140:in `service'
	/root/.rbenv/versions/2.6.9/lib/ruby/2.6.0/webrick/httpserver.rb:96:in `run'
	/root/.rbenv/versions/2.6.9/lib/ruby/2.6.0/webrick/server.rb:307:in `block in start_thread'