ADOT guidelines for testing and marking partner exporters as stable about aws-otel-community

[alolita]

Hi @MikeGoldsmith Please use the latest version of the upstream collector-contrib release to run your end-to-end tests for your exporter for traces. For example, if you are using the Honeycomb end-point, test Collector ingestion (OTLP) and export to Honeycomb's service endpoint.Hope that clarifies.

[alolita]

The latest Collector RC as of August 2 2021 can be found at https://github.com/open-telemetry/opentelemetry-collector/releases/tag/v0.31.0

[tigrannajaryan]

@alolita from what I see aws-otel-collector is still depending on Otel Collector v0.30.0. Do you plan to update to v0.31.0?
I believe once you do that the ADOT Testing Framework will run automatically and the results you are looking for will be in the Github actions, so you already did most of the hard work for all partner components, test steps 1-3 will be completed automatically.

For step 4:
a) CodeQL is run for all collector-contrib repo automatically, so provided that this passes we OK, right?
b) Similarly, if I am not wrong Gosec is also part of the linting on collector-contrib repo, so all partner components already pass this.

So, just to confirm, provided that the existing checks and tests that are in place pass, then that confirms the partner components are good and satisfy the requirements for steps 1-4. Is this correct or there is something additional you need?

[rmfitzpatrick]

@mraditya, @alolita any chance you'd be able to confirm that you will be adopting 0.31.0 in the aws-otel-collector project so that we're able to confirm all existing tests are successful for provided components in your CI env (~aws-observability/aws-otel-collector#600)? I'd be happy to contribute a PR for this that takes care of the required proto changes if desired.

[cartermp]

ACK on needing v0.31.0 - we will continue testing once it is available: https://github.com/honeycombio/telemetry-team/issues/69#issuecomment-896139534

[alolita]

Update on the Collector release and testing timelines:

Deliverables:

• 8/23 OTEL Collector core will be marked as stable
• 8/27 OTEL will announce tracing stability
• 8/27 OTEL Collector contrib components will be ready / marked stable to be included in Collector contrib release (includes AWS and partners exporters)
• 9/23 ADOT announces tracing stable GA with X-Ray, OTLP, and Partner tracing exporters

[alolita]

For partners using OTLP exporters to connect with their service end-points, please follow the steps below:

Testing includes:

• Use ADOT Collector to complete end-to-end testing with your service end-point which uses the OTLP protocol
• We are assuming that you are using your testing pipelines to test the ADOT Collector with your service end-point to ensure there are no performance degradations and security vulnerabilities.

How to do conduct testing

• Use the latest version of the Collector (corresponding to the tracing stable release which will be 0.33 as of 8/16)

Expected Result

• ADOT Stable Readiness issue: File an issue here to publish your test results for your end-to-end tests with OTLP <-> your service end-point.

Deliverables

• 8/23 OTEL Collector core will be marked as stable
• 8/27 OTEL will announce tracing stability
• 8/27 OTEL Collector contrib components will be ready / marked stable to be included in Collector contrib release (includes AWS and partners exporters)
• 9/23 ADOT announces tracing stable GA with X-Ray, OTLP, and Partner tracing exporters

[MikeGoldsmith]

@alolita please can you confirm for OTLP exporter partners exactly which tests you'd like us to run, and steps to execute them?

I've looked through the aws-collector and aws-collector-test-framework projects and am unsure what is expected for us to run. I have setup the test-framework locally and am able to execute the otlp_mock test case, but if I override the otconfig.tpl otlp_trace test case to export to Honeycomb, the test fails because the validator is unable to validate trace data was successfully processed.

Thanks

[MikeGoldsmith]

@alolita @mraditya any update on the above?