Metrics aren't being exported to Sumologic about sumologic-otel-collector

[natehitze]

Yes, that's definitely possible. I was just pushing whatever default automatic metric(s) that came out of the OTEL package I added to my project. I didn't see any notes about histograms not being supported so I never thought to check which metrics were being sent -- I was just trying to get something quick setup to test it. We ended up not choosing Sumo (or OTEL in general). Thanks for looking into this.

By the way, the install documentation was incredibly helpful. I think I spent more time getting TLS setup than installing and configuring the collector.

[swiatekm-sumo]

Your configuration looks correct to me. Can you post the full logs from otel startup? There should be information about what the extension is doing there.

Relatedly, can you see the collector in the Collectors list in your Sumo account?

[natehitze]

Sure thing, here it is. Yes the collector is in my Sumo account. Logs forward correctly from my application -> otelcol-sumo (this collector) -> Sumo. I'll include screenshots from Sumo below as well.

2022-08-05T13:20:46.848Z        info    service/telemetry.go:137        Serving Prometheus metrics      {"address": ":8888", "level": "basic"}
2022-08-05T13:20:46.848Z        info    components/components.go:30     Stability level of component is undefined      {"kind": "exporter", "data_type": "logs", "name": "sumologic", "stability": "undefined"}
2022-08-05T13:20:46.848Z        warn    [email protected]/exporter.go:81
***********************************************************************************************************************************************************
***    Translating attributes is deprecated and is going to be dropped soon. Please see the migration document:
                                    ***
***    https://github.com/SumoLogic/sumologic-otel-collector/blob/main/docs/Upgrading.md#sumologic-exporter-drop-support-for-translating-attributes.    ***
***********************************************************************************************************************************************************
        {"kind": "exporter", "data_type": "logs", "name": "sumologic"}
2022-08-05T13:20:46.851Z        warn    [email protected]/exporter.go:89
***********************************************************************************************************************************************************
***    Translating Telegraf metric names is deprecated and is going to be dropped soon. Please see the migration document:                              ***
***    https://github.com/SumoLogic/sumologic-otel-collector/blob/main/docs/Upgrading.md#sumologic-exporter-drop-support-for-translating-attributes.    ***
***********************************************************************************************************************************************************
        {"kind": "exporter", "data_type": "logs", "name": "sumologic"}
2022-08-05T13:20:46.851Z        info    [email protected]/exporter.go:119        Sumo Logic Exporter configured  {"kind": "exporter", "data_type": "logs", "name": "sumologic", "log_format": "otlp", "metric_format": "otlp", "trace_format": "otlp"}
2022-08-05T13:20:46.851Z        info    components/components.go:30     Stability level of component is undefined      {"kind": "exporter", "data_type": "metrics", "name": "sumologic", "stability": "undefined"}
2022-08-05T13:20:46.851Z        warn    [email protected]/exporter.go:81
***********************************************************************************************************************************************************
***    Translating attributes is deprecated and is going to be dropped soon. Please see the migration document:
                                    ***
***    https://github.com/SumoLogic/sumologic-otel-collector/blob/main/docs/Upgrading.md#sumologic-exporter-drop-support-for-translating-attributes.    ***
***********************************************************************************************************************************************************
        {"kind": "exporter", "data_type": "metrics", "name": "sumologic"}
2022-08-05T13:20:46.851Z        warn    [email protected]/exporter.go:89
***********************************************************************************************************************************************************
***    Translating Telegraf metric names is deprecated and is going to be dropped soon. Please see the migration document:                              ***
***    https://github.com/SumoLogic/sumologic-otel-collector/blob/main/docs/Upgrading.md#sumologic-exporter-drop-support-for-translating-attributes.    ***
***********************************************************************************************************************************************************
        {"kind": "exporter", "data_type": "metrics", "name": "sumologic"}
2022-08-05T13:20:46.851Z        info    [email protected]/exporter.go:119        Sumo Logic Exporter configured  {"kind": "exporter", "data_type": "metrics", "name": "sumologic", "log_format": "otlp", "metric_format": "otlp", "trace_format": "otlp"}
2022-08-05T13:20:46.851Z        info    components/components.go:30     In development component. May change in the future.     {"kind": "exporter", "data_type": "metrics", "name": "logging", "stability": "in development"}
2022-08-05T13:20:46.852Z        info    components/components.go:30     Stability level of component is undefined      {"kind": "exporter", "data_type": "traces", "name": "sumologic", "stability": "undefined"}
2022-08-05T13:20:46.852Z        warn    [email protected]/exporter.go:81
***********************************************************************************************************************************************************
***    Translating attributes is deprecated and is going to be dropped soon. Please see the migration document:
                                    ***
***    https://github.com/SumoLogic/sumologic-otel-collector/blob/main/docs/Upgrading.md#sumologic-exporter-drop-support-for-translating-attributes.    ***
***********************************************************************************************************************************************************
        {"kind": "exporter", "data_type": "traces", "name": "sumologic"}
2022-08-05T13:20:46.852Z        warn    [email protected]/exporter.go:89
***********************************************************************************************************************************************************
***    Translating Telegraf metric names is deprecated and is going to be dropped soon. Please see the migration document:                              ***
***    https://github.com/SumoLogic/sumologic-otel-collector/blob/main/docs/Upgrading.md#sumologic-exporter-drop-support-for-translating-attributes.    ***
***********************************************************************************************************************************************************
        {"kind": "exporter", "data_type": "traces", "name": "sumologic"}
2022-08-05T13:20:46.852Z        info    [email protected]/exporter.go:119        Sumo Logic Exporter configured  {"kind": "exporter", "data_type": "traces", "name": "sumologic", "log_format": "otlp", "metric_format": "otlp", "trace_format": "otlp"}
2022-08-05T13:20:46.852Z        info    extensions/extensions.go:42     Starting extensions...
2022-08-05T13:20:46.852Z        info    extensions/extensions.go:45     Extension is starting...        {"kind": "extension", "name": "sumologic"}
2022-08-05T13:20:46.853Z        info    credentials/credentialsstore_localfs.go:146     Collector registration credentials retrieved from local fs      {"kind": "extension", "name": "sumologic", "path": "/root/.sumologic-otel-collector/7d22c4b43d0336303452c0d9ecd356b1dbfb149651c3117a436a20de7ea4fbbb"}
2022-08-05T13:20:46.856Z        info    [email protected]/extension.go:188      Validating collector credentials...     {"kind": "extension", "name": "sumologic", "collector_credential_id": "AnTNXeXKT2PecxZIkAaU", "collector_id": "0000000010C3388E"}
2022-08-05T13:20:47.430Z        info    [email protected]/extension.go:255      Found stored credentials, skipping registration {"kind": "extension", "name": "sumologic", "collector_name": "myproduct"}
2022-08-05T13:20:47.430Z        info    extensions/extensions.go:49     Extension started.      {"kind": "extension", "name": "sumologic"}
2022-08-05T13:20:47.430Z        info    extensions/extensions.go:45     Extension is starting...        {"kind": "extension", "name": "file_storage"}
2022-08-05T13:20:47.430Z        info    extensions/extensions.go:49     Extension started.      {"kind": "extension", "name": "file_storage"}
2022-08-05T13:20:47.430Z        info    pipelines/pipelines.go:74       Starting exporters...
2022-08-05T13:20:47.430Z        info    pipelines/pipelines.go:78       Exporter is starting... {"kind": "exporter", "data_type": "logs", "name": "sumologic"}
2022-08-05T13:20:47.430Z        info    [email protected]/extension.go:490      Heartbeat loop initialized. Starting to send hearbeat requests  {"kind": "extension", "name": "sumologic", "collector_name": "myproduct", "collector_id": "0000000010C3388E"}
2022-08-05T13:20:47.438Z        info    pipelines/pipelines.go:82       Exporter started.       {"kind": "exporter", "data_type": "logs", "name": "sumologic"}
2022-08-05T13:20:47.438Z        info    pipelines/pipelines.go:78       Exporter is starting... {"kind": "exporter", "data_type": "metrics", "name": "sumologic"}
2022-08-05T13:20:47.438Z        info    pipelines/pipelines.go:82       Exporter started.       {"kind": "exporter", "data_type": "metrics", "name": "sumologic"}
2022-08-05T13:20:47.438Z        info    pipelines/pipelines.go:78       Exporter is starting... {"kind": "exporter", "data_type": "metrics", "name": "logging"}
2022-08-05T13:20:47.438Z        info    pipelines/pipelines.go:82       Exporter started.       {"kind": "exporter", "data_type": "metrics", "name": "logging"}
2022-08-05T13:20:47.438Z        info    pipelines/pipelines.go:78       Exporter is starting... {"kind": "exporter", "data_type": "traces", "name": "sumologic"}
2022-08-05T13:20:47.438Z        info    pipelines/pipelines.go:82       Exporter started.       {"kind": "exporter", "data_type": "traces", "name": "sumologic"}
2022-08-05T13:20:47.438Z        info    pipelines/pipelines.go:86       Starting processors...
2022-08-05T13:20:47.438Z        info    pipelines/pipelines.go:98       Starting receivers...
2022-08-05T13:20:47.438Z        info    pipelines/pipelines.go:102      Receiver is starting... {"kind": "receiver", "name": "otlp", "pipeline": "logs"}
2022-08-05T13:20:47.441Z        info    otlpreceiver/otlp.go:70 Starting GRPC server on endpoint 0.0.0.0:4317   {"kind": "receiver", "name": "otlp", "pipeline": "logs"}
2022-08-05T13:20:47.453Z        info    pipelines/pipelines.go:106      Receiver started.       {"kind": "receiver", "name": "otlp", "pipeline": "logs"}
2022-08-05T13:20:47.453Z        info    pipelines/pipelines.go:102      Receiver is starting... {"kind": "receiver", "name": "otlp", "pipeline": "metrics"}
2022-08-05T13:20:47.453Z        info    pipelines/pipelines.go:106      Receiver started.       {"kind": "receiver", "name": "otlp", "pipeline": "metrics"}
2022-08-05T13:20:47.453Z        info    pipelines/pipelines.go:102      Receiver is starting... {"kind": "receiver", "name": "otlp", "pipeline": "traces"}
2022-08-05T13:20:47.453Z        info    pipelines/pipelines.go:106      Receiver started.       {"kind": "receiver", "name": "otlp", "pipeline": "traces"}
2022-08-05T13:20:47.453Z        info    service/collector.go:215        Starting otelcol-sumo...        {"Version": "v0.56.0-sumo-0", "NumCPU": 2}
2022-08-05T13:20:47.453Z        info    service/collector.go:128        Everything is ready. Begin running and processing data.
2022-08-05T13:20:53.542Z      info    service/collector.go:159        Received signal from OS {"signal": "interrupt"}
2022-08-05T13:20:53.542Z        info    service/collector.go:231        Starting shutdown...
2022-08-05T13:20:53.542Z        info    pipelines/pipelines.go:118      Stopping receivers...
2022-08-05T13:20:53.542Z        info    pipelines/pipelines.go:125      Stopping processors...
2022-08-05T13:20:53.542Z        info    pipelines/pipelines.go:132      Stopping exporters...
2022-08-05T13:20:53.542Z        info    extensions/extensions.go:56     Stopping extensions...
2022-08-05T13:20:53.542Z        info    [email protected]/extension.go:495      Heartbeat sender turned off     {"kind": "extension", "name": "sumologic", "collector_name": "myproduct", "collector_id": "0000000010C3388E"}

[swiatekm-sumo]

Just to make sure we're not getting wires crossed here, your screenshot shows the collector name to end in -otel whereas in the log the name is myproduct. Is the latter value just an example?

[natehitze]

Yes, sorry! They are the same collector name for sure, I just masked them differently.

[swiatekm-sumo]

Is it possible that you're only pushing histograms? Sumo doesn't support Histograms via OTLP yet, and I can only see histograms in your log. Can you try with something simple like the hostmetrics receiver and see if data shows up then?