aspecto-io / opentelemetry-ext-js Goto Github PK
View Code? Open in Web Editor NEWjs extensions for the open-telemetry project
Home Page: https://www.aspecto.io
License: Apache License 2.0
js extensions for the open-telemetry project
Home Page: https://www.aspecto.io
License: Apache License 2.0
The KafkaJS instrumentation is not setting any headers on the messages.
Versions:
"@opentelemetry/api": "^1.0.3",
"@opentelemetry/core": "^1.0.0",
"@opentelemetry/exporter-jaeger": "^1.0.0",
"@opentelemetry/exporter-prometheus": "^0.27.0",
"@opentelemetry/instrumentation": "^0.27.0",
"@opentelemetry/instrumentation-express": "^0.27.0",
"@opentelemetry/instrumentation-http": "^0.27.0",
"@opentelemetry/resources": "^1.0.0",
"@opentelemetry/sdk-node": "^0.27.0",
"@opentelemetry/sdk-trace-base": "^1.0.0",
"@opentelemetry/sdk-metrics-base": "^0.27.0",
"@opentelemetry/sdk-trace-node": "^1.0.0",
"@opentelemetry/semantic-conventions": "^1.0.1",
"opentelemetry-instrumentation-elasticsearch": "^0.27.1",
"opentelemetry-instrumentation-kafkajs": "^0.27.1",
"@opentelemetry/propagator-b3": "^1.2.0",
"@opentelemetry/propagator-jaeger": "^1.2.0",
Hey, great work on the instrumentations!
Do you have plans to upstream these to opentelemetry-js-contrib
in the nearer future?
Those 2 functions are executing queries on the database.
The queries create sequelize spans that are missing the context of what happened:
SELECT 1+1 AS result
)I suggest to instrument these functions as well, so the instrumentation user will know that a call to authenticate
or model.sync
happened.
First of all thanks for the great job you are doing!
Are there any plans on supporting instrumentation for the client side of socket.io?
Hey, it would be amazing if this instrumentation will be maintained by otel community :)
Hi,
I'm not sure what I'm doing wrong and I couldn't find any extra doc
I can't get the typeorm or kafkajs instrumentation to work. I running with NestJS and kafkajs wrapping is implemented in another lib I created.
When running only the official mysql2 instrumentation It is working and showing the queries.
The documentation states that I need to disable internal plugin, but NodeTracerProvider
configuration does not contain such a member named plugin
I would really much appreciate your help!
This is my setup:
import { JaegerExporter } from '@opentelemetry/exporter-jaeger';
import { registerInstrumentations } from '@opentelemetry/instrumentation';
import { ExpressInstrumentation } from '@opentelemetry/instrumentation-express';
import { HttpInstrumentation } from '@opentelemetry/instrumentation-http';
import { IORedisInstrumentation } from '@opentelemetry/instrumentation-ioredis';
import { NestInstrumentation } from '@opentelemetry/instrumentation-nestjs-core';
import { B3Propagator } from '@opentelemetry/propagator-b3';
import { Resource } from '@opentelemetry/resources';
import { SimpleSpanProcessor } from '@opentelemetry/sdk-trace-base';
import { NodeTracerProvider } from '@opentelemetry/sdk-trace-node';
import { SemanticResourceAttributes } from '@opentelemetry/semantic-conventions';
import { KafkaJsInstrumentation } from 'opentelemetry-instrumentation-kafkajs';
import { TypeormInstrumentation } from 'opentelemetry-instrumentation-typeorm';
const provider = new NodeTracerProvider({
resource: new Resource({
[SemanticResourceAttributes.SERVICE_NAME]: 'my-service',
}),
});
const exporter = new JaegerExporter();
provider.addSpanProcessor(new SimpleSpanProcessor(exporter));
provider.register({propagator: new B3Propagator()});
registerInstrumentations({
instrumentations: [
new ExpressInstrumentation(),
new HttpInstrumentation(),
// new MySQL2Instrumentation(), <--- working
new IORedisInstrumentation(),
new KafkaJsInstrumentation(), <--- not working
new NestInstrumentation(), <--- not working
new TypeormInstrumentation() <--- not working
]
});
I'm using this package.
https://github.com/aspecto-io/opentelemetry-ext-js/tree/master/packages/instrumentation-kafkajs
Default instrumentation setup. I send a kafka message with headers traceparent .. and baggage. The baggage isn't injected into context.
const baggage = propagation.getBaggage(context.active()); // baggage is undefined
traceId, spanId is fine with traceparent header..
Thoughts ?
patching EventEmitter is supported by opentelemetry by calling:
context.bind(o, context);
where o is an instanceof EventEmitter.
For some reason, Request
object from aws-sdk package does not fall into this category, thus context is not binded to it's events.
This made me patch the request.on('complete'
event manually in the _registerCompletedEvent
which works fine, but it's better to cover all cases and use the available infra.
Hi,
As of today, it appears that only the EntityManager
is instrumented according to the diagnostics logs, which leaves some part of my microservices "uninstrumented" (I still have the PG related instrumentationwhich is fine for now).
Do you have any plan supporting QueryBuilder
s and Repository
in the (near) future ?
If it is not a priority on your end, and due to the fact that I am pretty new to Open Telemetry JS, do you have any guidelines to share on how to achieve it myself for an eventual MR ?
Thanks,
First, i try to use this https://github.com/aspecto-io/opentelemetry-ext-js/tree/master/packages/instrumentation-kafkajs , but not initialize on my app.
After that i try to make custom instrumentation
Here is the repo for reproduction https://github.com/ruifernando7/otel-kafka
When i try to start the app (using yarn items
), the init
function is trigger.
But this patch function is not trigger
https://github.com/ruifernando7/otel-kafka/blob/50b4227eb7f13987dd39088b988fedb5c19c4084/src/kafka-instrumentation/kafka.ts#L16-L20
When I try to use another library like 'ws', the patch function is triggered, but no luck on kafkajs. Why is that?
The patch function is trigger when using kafkajs.
The patch function didn't trigger when using kafkajs
An application using sequelize over postgres.
Opentelemetry setup using instrumentation-sequelize and instrumentation-pg.
Have the sequelize span as the parent of postgres span
I'm trying to setup typeorm instrumentation following way:
export const otelSDK = new NodeSDK({
metricInterval: 1000,
traceExporter: exporter,
textMapPropagator: new CompositePropagator({
propagators: [
new W3CTraceContextPropagator(),
new W3CBaggagePropagator(),
new B3Propagator(),
new B3Propagator({
injectEncoding: B3InjectEncoding.MULTI_HEADER,
}),
],
}),
spanProcessor: new BatchSpanProcessor(exporter, {
maxQueueSize: 10,
maxExportBatchSize: 10,
scheduledDelayMillis: 50,
exportTimeoutMillis: 3000,
}),
instrumentations: [
getNodeAutoInstrumentations(),
new TypeormInstrumentation({
collectParameters: true,
enabled: true,
path: 'opentelemetry-plugin-better-sqlite3',
}),
],
});
I've enabled logger for all logs
diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.ALL);
Once I launch application there is no typeorm instrumentation loaded
@opentelemetry/api: Registered a global for diag v1.0.4.
Loading instrumentation for @opentelemetry/instrumentation-amqplib
Loading instrumentation for @opentelemetry/instrumentation-aws-lambda
No modules instrumentation has been defined, nothing will be patched
Loading instrumentation for @opentelemetry/instrumentation-aws-sdk
Loading instrumentation for @opentelemetry/instrumentation-bunyan
Loading instrumentation for @opentelemetry/instrumentation-cassandra-driver
Loading instrumentation for @opentelemetry/instrumentation-connect
Loading instrumentation for @opentelemetry/instrumentation-dns
Loading instrumentation for @opentelemetry/instrumentation-express
Loading instrumentation for @opentelemetry/instrumentation-fastify
Loading instrumentation for @opentelemetry/instrumentation-generic-pool
Loading instrumentation for @opentelemetry/instrumentation-graphql
Loading instrumentation for @opentelemetry/instrumentation-grpc
Loading instrumentation for @opentelemetry/instrumentation-hapi
Loading instrumentation for @opentelemetry/instrumentation-http
Loading instrumentation for @opentelemetry/instrumentation-ioredis
Loading instrumentation for @opentelemetry/instrumentation-knex
Loading instrumentation for @opentelemetry/instrumentation-koa
Loading instrumentation for @opentelemetry/instrumentation-memcached
Loading instrumentation for @opentelemetry/instrumentation-mongodb
Loading instrumentation for @opentelemetry/instrumentation-mysql2
Loading instrumentation for @opentelemetry/instrumentation-mysql
Loading instrumentation for @opentelemetry/instrumentation-nestjs-core
Loading instrumentation for @opentelemetry/instrumentation-net
Loading instrumentation for @opentelemetry/instrumentation-pg
Loading instrumentation for @opentelemetry/instrumentation-pino
Loading instrumentation for @opentelemetry/instrumentation-redis
Loading instrumentation for @opentelemetry/instrumentation-restify
Loading instrumentation for @opentelemetry/instrumentation-winston
TypeORM is in version 0.2.45
"opentelemetry-instrumentation-typeorm": "^0.27.1",
"opentelemetry-plugin-better-sqlite3": "^0.8.1",
"@opentelemetry/instrumentation": "^0.28.0",
"@opentelemetry/sdk-node": "^0.27.0",
"@opentelemetry/sdk-trace-base": "^1.0.1",
"@opentelemetry/sdk-trace-node": "^1.2.0",
"@opentelemetry/tracing": "^0.24.0",
No spans/traces are recorded for TypeORM in Jaeger.
Hello,
I have a Java service(producer) and Nodejs service(consumer) where they are connecting to Kafka. Java service sends a JSON object to Kafka and Node.js service consumes it and saves it to the database.
I am using b3-propagators multi
and when I made a comparison to traces between services I am aware that the Kafka one has a different one where it breaks the traces.If you check the consumer message and java the traces are same which is 2721916b8dbb8e1d58d6d269a10b056d
where kafka one 2273c5c3fe7cd6edc27f3c3c8366ab7e
.. Why this traceID changing? Is it due to b3 propagation? What might be the root cause of tracing is breaking?
Node.js(consumer)
{
partition: 0,
offset: '0',
value: '{"title":"shoes","desc":"shoes","img":"test4.svg","categories":["shoes","man"],"size":"M","price":"50"}',
headers: {
'X-B3-TraceId': '2721916b8dbb8e1d58d6d269a10b056d',
'X-B3-SpanId': '4e0c3513fa73ac8d',
'X-B3-Sampled': '1',
traceparent: '00-2721916b8dbb8e1d58d6d269a10b056d-4e0c3513fa73ac8d-01',
__TypeId__: 'com.basketservice.model.Products'
}
}
{
traceId: '2273c5c3fe7cd6edc27f3c3c8366ab7e',
parentId: undefined,
name: 'productCreated',
id: '69392d37aade564a',
kind: 4,
timestamp: 1659635058364999,
duration: 36058,
attributes: {
'messaging.system': 'kafka',
'messaging.destination': 'productCreated',
'messaging.destination_kind': 'topic',
'messaging.operation': 'process'
},
status: { code: 0 },
events: [],
links: []
}
{
traceId: '2273c5c3fe7cd6edc27f3c3c8366ab7e',
parentId: '69392d37aade564a',
name: 'mongodb.insert',
id: '9ebe87bfc3bb5fbc',
kind: 2,
timestamp: 1659635058382611,
duration: 16705,
attributes: {
'db.system': 'mongodb',
'db.name': 'products',
'db.mongodb.collection': 'products',
'net.host.name': '192.168.64.4',
'net.host.port': '27017',
'db.statement': '{"title":"?","desc":"?","img":"?","categories":"?","size":"?","price":"?","_id":"?","createdAt":"?","updatedAt":"?","__v":"?"}'
},
status: { code: 0 },
events: [],
links: []
}
Java(producer)
2022-08-04 17:44:17.314 trace_id=2721916b8dbb8e1d58d6d269a10b056d span_id=dc350ce3af0132d2 trace_flags=01 INFO 7 --- [ctor-http-nio-3] c.a.b.controller.BasketController : PRODUCT ADDED Products(title=shoes, desc=shoes, img=test4.svg, categories=[shoes, man], size=M, price=50)
2022-08-04 17:44:17.438 trace_id=2721916b8dbb8e1d58d6d269a10b056d span_id=dc350ce3af0132d2 trace_flags=01 INFO 7 --- [ctor-http-nio-3] o.a.k.clients.producer.ProducerConfig : ProducerConfig values:
Here is tracing.js
const { getNodeAutoInstrumentations } = require("@opentelemetry/auto-instrumentations-node");
const { registerInstrumentations } = require('@opentelemetry/instrumentation');
const opentelemetry = require("@opentelemetry/sdk-node");
const api = require('@opentelemetry/api');
const { CompositePropagator } = require('@opentelemetry/core');
const { Resource } = require('@opentelemetry/resources');
const { SemanticResourceAttributes } = require('@opentelemetry/semantic-conventions');
const { B3Propagator, B3InjectEncoding } = require('@opentelemetry/propagator-b3');
const {ConsoleSpanExporter} = require('@opentelemetry/sdk-trace-base');
// const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-grpc');
const { ExpressInstrumentation } = require('@opentelemetry/instrumentation-express');
const { KafkaJsInstrumentation } = require('opentelemetry-instrumentation-kafkajs');
const serviceName = 'product-service'
// const traceExporter = new OTLPTraceExporter({
// url: process.env.OTEL_EXPORTER_OTLP_TRACES_ENDPOINT || 'http://otel-collector:4317/'
// });
api.propagation.setGlobalPropagator(
new CompositePropagator({
propagators: [
new B3Propagator({ injectEncoding: B3InjectEncoding.MULTI_HEADER }),
],
})
);
const sdk = new opentelemetry.NodeSDK({
resource: new Resource({
[SemanticResourceAttributes.SERVICE_NAME]: serviceName,
}),
traceExporter: new ConsoleSpanExporter(),
instrumentations: [getNodeAutoInstrumentations()]
});
registerInstrumentations({
instrumentations: [
new ExpressInstrumentation(),
new KafkaJsInstrumentation()
],
});
sdk
.start()
....
Here are the dependencies
"dependencies": {
"@opentelemetry/api": "^1.1.0",
"@opentelemetry/auto-instrumentations-node": "^0.31.0",
"@opentelemetry/exporter-trace-otlp-grpc": "^0.30.0",
"@opentelemetry/instrumentation-express": "^0.30.0",
"@opentelemetry/propagator-b3": "^1.4.0",
"@opentelemetry/resources": "^1.4.0",
"@opentelemetry/sdk-node": "^0.30.0",
"@opentelemetry/semantic-conventions": "^1.4.0",
"axios": "^0.27.2",
"express": "^4.18.1",
"kafkajs": "^2.1.0",
"mongoose": "^6.4.6",
"opentelemetry-instrumentation-kafkajs": "^0.29.0",
"prom-client": "^14.0.1"
}
When calling Mongoose operations that have no callback, and their response itself is not a promise object, the supplied responseHook set during configuration is not applied to emitted spans.
I believe this is because applyResponseHook()
function is called after the span.end() call executes, meaning any modifications done in the supplied responseHook configuration parameter are not applied to the span emitted for that operation.
See
opentelemetry-ext-js/packages/instrumentation-mongoose/src/utils.ts
Lines 82 to 86 in 8d42dac
I believe the correct order should be:
if (!(execResponse instanceof Promise)) {
applyResponseHook(span, execResponse, responseHook);
span.end();
return execResponse;
}
const { NodeTracerProvider } = require("opentelemetry-plugin-kafkajs");
Throws
TypeError: NodeTracerProvider is not a constructor
This is how I finally got it working.
import { NodeTracerProvider } from '@opentelemetry/node';
import { trace, Tracer } from '@opentelemetry/api';
import { SimpleSpanProcessor, SpanExporter } from '@opentelemetry/tracing';
import { JaegerExporter } from '@opentelemetry/exporter-jaeger';
export const getTracer = (serviceName: string): Tracer => {
const provider = new NodeTracerProvider({
plugins: {
kafkajs: {
enabled: true,
path: 'opentelemetry-plugin-kafkajs',
},
},
});
const exporter: SpanExporter = new JaegerExporter({
serviceName,
});
provider.addSpanProcessor(new SimpleSpanProcessor(exporter));
provider.register();
return trace.getTracer('kafkajs-example');
};
Opentelemetry released a new version of the base instrumentation and SDK. I would like to help but I know very little about how these packages work. Instead, I will leave the PR from the contrib packages as a reference:
open-telemetry/opentelemetry-js-contrib#1054
Here are the changelogs for the new releases:
https://github.com/open-telemetry/opentelemetry-js/releases/tag/experimental%2Fv0.29.0
https://github.com/open-telemetry/opentelemetry-js/releases/tag/v1.3.0
Opentelemetry released a new version of the base instrumentation and SDK. I would like to help but I know very little about how these packages work. Instead, I will leave the PR from the contrib packages as a reference:
open-telemetry/opentelemetry-js-contrib#984
Here are the changelogs for the new releases:
https://github.com/open-telemetry/opentelemetry-js/releases/tag/v0.28.0
https://github.com/open-telemetry/opentelemetry-js/releases/tag/v1.2.0
Thank you for building these instrumentation libraries!
When looking at the instrumentation libraries in https://github.com/open-telemetry/opentelemetry-js-contrib/tree/main/plugins/node, I found it very useful to see the supported versions listed for each library (i.e., the versions of foo
supported by @opentelemetry/instrumentation-foo
).
Would it be possible to add something similar to the libraries in this repo? I'm especially curious if the Mongoose instrumentation supports Mongoose v6.
Needs to investigate why daily tests are failing
https://github.com/aspecto-io/opentelemetry-ext-js/actions/runs/1470162456
When connecting to AMQP via URL, the whole URL gets sent in span attributes, potentially exposing username and password. Is it possible to censor it or to exclude the field?
Example result:
Elasticsearch instrumentation doesn't support @elastic/elasticsearch
latest version - 8.0.0
Hi,
I noticed that [email protected]
is just logging the statement, which might have parameter placeholders (e.g. $1
, $2
, etc). I would like to add these parameters to the span attributes. An example is INSERT INTO "addresses" ("id","city","lineOne","zipCode","state","createdAt","updatedAt") VALUES ($1,$2,$3,$4,$5,$6,$7);
.
As a more general mechanism, perhaps a new config.queryHook(span, sql, option)
(or something like that) could be created, allowing the service author (me ๐ ) to add span attributes based on the sql.
(If you'd like I can attempt a PR to implement this, following the pattern of config.responseHook(span, response)
.)
I have 2 services, a producer and consumer communicating over sqs.
I am using https://github.com/aspecto-io/opentelemetry-ext-js/tree/master/packages/instrumentation-aws-sdk as well as https://www.npmjs.com/package/sqs-consumer, but I am not loading instrumentation-aws-sdk within sqs-consumer, Iโm loading it through the consumer service (that is using sqs-consumer)
I am getting spans to export. aws-sdk instrumentation is clearly somewhat working, or I would not expect to see any spans at all. But I am losing context propagation. The producer is correctly placing traceparent in the message attributes.
Iโm posting here as a sanity check:
Version info:
"aws-sdk": "^2.699.0",
"sqs-consumer": "^5.4.0",
"opentelemetry-instrumentation-aws-sdk": "^0.24.2",
I have been using pg instrumentation for a while and wanted to see what I could get out of sequelize instrumentation but when I used it, I got no new traces.
Am I missing something ?
Sequelize v5, opentelemetry v 0.19
Same case as #169 - since Lightstep is interested in having Elasticsearch instrumentation in OTel, we were originally thinking of 'porting' (i.e. duplicating) the code from this repo (given it has Apache License) to opentelemetry-js-contrib, but happy to help in any way - once a decision is made in #169, that is.
I have two different services where Java(producer) and Node.js(consumer). I am sending JSON object to Kafka from Java service where Node.js service consumes it and save it to the database.
I tried to use opentelemetry-instrumentation-kafkajs
as below but I could not able to see any traces between my Node.js service and Kafka.
Here are my container logs when I hit the POST request from Java. It can able to consume and save to database without issue but I could not see any traces generated where I am expecting to see.
What I am doing wrong? What is the root cause of this?
Dependencies and versions
Kafka: v7.2.1
"dependencies": {
"@opentelemetry/api": "^1.1.0",
"@opentelemetry/auto-instrumentations-node": "^0.31.0",
"@opentelemetry/exporter-trace-otlp-grpc": "^0.30.0",
"@opentelemetry/instrumentation-express": "^0.30.0",
"@opentelemetry/propagator-b3": "^1.4.0",
"@opentelemetry/resources": "^1.4.0",
"@opentelemetry/sdk-node": "^0.30.0",
"@opentelemetry/semantic-conventions": "^1.4.0",
"axios": "^0.27.2",
"dotenv": "^16.0.1",
"express": "^4.18.1",
"kafkajs": "^2.1.0",
"mongoose": "^6.4.6",
"opentelemetry-instrumentation-kafkajs": "^0.29.0",
"prom-client": "^14.0.1"
Application logs
docker logs -f 614c0f9c9f7b
> [email protected] start
> node --require './src/tracing.js' src/index.js
Product service listening port 8005
Tracing initialized for product-service
{"level":"ERROR","timestamp":"2022-08-04T11:01:27.879Z","logger":"kafkajs","message":"[Connection] Connection error: connect ECONNREFUSED 172.23.0.6:9092","broker":"kafka:9092","clientId":"product-service","stack":"Error: connect ECONNREFUSED 172.23.0.6:9092\n at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1187:16)"}
{"level":"ERROR","timestamp":"2022-08-04T11:01:27.880Z","logger":"kafkajs","message":"[BrokerPool] Failed to connect to seed broker, trying another broker from the list: Connection error: connect ECONNREFUSED 172.23.0.6:9092","retryCount":0,"retryTime":287}
Connected to mongod
{"level":"ERROR","timestamp":"2022-08-04T11:01:28.170Z","logger":"kafkajs","message":"[Connection] Connection error: connect ECONNREFUSED 172.23.0.6:9092","broker":"kafka:9092","clientId":"product-service","stack":"Error: connect ECONNREFUSED 172.23.0.6:9092\n at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1187:16)"}
{"level":"ERROR","timestamp":"2022-08-04T11:01:28.170Z","logger":"kafkajs","message":"[BrokerPool] Failed to connect to seed broker, trying another broker from the list: Connection error: connect ECONNREFUSED 172.23.0.6:9092","retryCount":1,"retryTime":638}
{"level":"ERROR","timestamp":"2022-08-04T11:01:28.810Z","logger":"kafkajs","message":"[Connection] Connection error: connect ECONNREFUSED 172.23.0.6:9092","broker":"kafka:9092","clientId":"product-service","stack":"Error: connect ECONNREFUSED 172.23.0.6:9092\n at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1187:16)"}
{"level":"ERROR","timestamp":"2022-08-04T11:01:28.811Z","logger":"kafkajs","message":"[BrokerPool] Failed to connect to seed broker, trying another broker from the list: Connection error: connect ECONNREFUSED 172.23.0.6:9092","retryCount":2,"retryTime":1042}
{"level":"ERROR","timestamp":"2022-08-04T11:01:29.855Z","logger":"kafkajs","message":"[Connection] Connection error: connect ECONNREFUSED 172.23.0.6:9092","broker":"kafka:9092","clientId":"product-service","stack":"Error: connect ECONNREFUSED 172.23.0.6:9092\n at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1187:16)"}
{"level":"ERROR","timestamp":"2022-08-04T11:01:29.855Z","logger":"kafkajs","message":"[BrokerPool] Failed to connect to seed broker, trying another broker from the list: Connection error: connect ECONNREFUSED 172.23.0.6:9092","retryCount":3,"retryTime":1908}
{"level":"ERROR","timestamp":"2022-08-04T11:01:31.828Z","logger":"kafkajs","message":"[Connection] Response Metadata(key: 3, version: 6)","broker":"kafka:9092","clientId":"product-service","error":"There is no leader for this topic-partition as we are in the middle of a leadership election","correlationId":1,"size":84}
{"level":"INFO","timestamp":"2022-08-04T11:01:32.101Z","logger":"kafkajs","message":"[Consumer] Starting","groupId":"products"}
{"level":"ERROR","timestamp":"2022-08-04T11:01:32.156Z","logger":"kafkajs","message":"[Connection] Response GroupCoordinator(key: 10, version: 2)","broker":"kafka:9092","clientId":"product-service","error":"The group coordinator is not available","correlationId":3,"size":55}
{"level":"ERROR","timestamp":"2022-08-04T11:01:32.511Z","logger":"kafkajs","message":"[Connection] Response GroupCoordinator(key: 10, version: 2)","broker":"kafka:9092","clientId":"product-service","error":"The group coordinator is not available","correlationId":4,"size":55}
{"level":"INFO","timestamp":"2022-08-04T11:01:36.223Z","logger":"kafkajs","message":"[ConsumerGroup] Consumer has joined the group","groupId":"products","memberId":"product-service-d1fb90db-1a61-4ecc-ba5e-7c6b3de6da22","leaderId":"product-service-d1fb90db-1a61-4ecc-ba5e-7c6b3de6da22","isLeader":true,"memberAssignment":{"productCreated":[0]},"groupProtocol":"RoundRobinAssigner","duration":4107}
{
partition: 0,
offset: '0',
value: '{"title":"Arsenal Third Shirt","desc":"Womens Footbal","img":"test2.svg","categories":["tshirt","woman"],"size":"S","price":"90"}'
}
Here is tracing.js
const { getNodeAutoInstrumentations } = require("@opentelemetry/auto-instrumentations-node");
const { registerInstrumentations } = require('@opentelemetry/instrumentation');
const opentelemetry = require("@opentelemetry/sdk-node");
const api = require('@opentelemetry/api');
const { CompositePropagator } = require('@opentelemetry/core');
const { Resource } = require('@opentelemetry/resources');
const { SemanticResourceAttributes } = require('@opentelemetry/semantic-conventions');
const { B3Propagator, B3InjectEncoding } = require('@opentelemetry/propagator-b3');
const { PinoInstrumentation } = require('@opentelemetry/instrumentation-pino');
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-grpc')
const { NodeTracerProvider } = require('@opentelemetry/sdk-trace-node');
const { HttpInstrumentation } = require('@opentelemetry/instrumentation-http');
const { ExpressInstrumentation } = require('@opentelemetry/instrumentation-express');
const { KafkaJsInstrumentation } = require('opentelemetry-instrumentation-kafkajs');
const serviceName = 'product-service'
const tracerProvider = new NodeTracerProvider({
resource: new Resource({
[SemanticResourceAttributes.SERVICE_NAME]: serviceName,
}),
plugins: {
kafkajs: { enabled: false, path: 'opentelemetry-plugin-kafkajs' }
}
});
const traceExporter = new OTLPTraceExporter({
url: process.env.OTEL_EXPORTER_OTLP_TRACES_ENDPOINT || 'http://otel-collector:4317/'
});
api.propagation.setGlobalPropagator(
new CompositePropagator({
propagators: [
new B3Propagator(),
new B3Propagator({ injectEncoding: B3InjectEncoding.MULTI_HEADER }),
],
})
);
const sdk = new opentelemetry.NodeSDK({
resource: new Resource({
[SemanticResourceAttributes.SERVICE_NAME]: serviceName,
}),
traceExporter: traceExporter,
instrumentations: [getNodeAutoInstrumentations()]
});
registerInstrumentations({
tracerProvider,
instrumentations: [
new PinoInstrumentation(),
new HttpInstrumentation(),
new ExpressInstrumentation(),
new KafkaJsInstrumentation()
],
});
sdk
.start()
.then(() => {
console.log('Tracing initialized for product-service')
})
.catch((error) => {
console.log('Error initializing tracing for product-service', error)
})
process.on('SIGTERM', () => {
sdk
.shutdown()
.then(() => {
console.log('Tracing terminated for product-service')
})
.catch((error) => {
console.log('Error terminating tracing for product-service', error)
})
.finally(() => {
process.exit(0)
})
})
Currently, we need to manually lint / prettier before pushing to github. This is done by running yarn prettier
.
The job can be done automatically by husky git hook, but it was removed from the repo due to a bug in husky with monorepos: typicode/husky#677
I now read in the issue that it might be fixed in husky v6.
This issue is to test and add the husky hook if it now works correctly.
Those instrumentations always have an underlying operation.
For example, if using postgres and having @opentelemetry/instrumentation-pg installed, we will end up with at least 2 spans for each operation.
Similar use case like:
I was trying to setup a simple trace that utilizes a SQS queue hoping that the trace would go through the sender and the receiver. However, the trace appears to be breaking upon receiving the message in the queue. The code that I used can be found here.
I was hoping the following operations would fall all under a single trace.
Can you please explain what I'm doing wrong and what am I missing? Or if this is even possible as I'm still new to open telemetry.
Any help is appreciated, Thanks! ๐
Hi guys,
I'm having some issues with several services that consume kafka messages but creates a new trace instead of connecting it to an existing one. This is not the case for all services, where the instrumentation is working as expected.
I'm using 0.27.X libs on all services which makes everything even more confusing.
How can I debug the issue?
Thank you
When converting from Jaeger Proto to OTEL readable span - in convertJaegerTagsToAttributes function - we currently convert all the tags to OTEL attributes.
We need to filter out the ones that are not supported by OTEL Specification.
Version 2.0.0 of the kafkajs library has been released (release notes)
Can you add support for this version?
I'm reaching out because you guys already wrote few instrumentation package for nodejs and i believe it's best to let you know that we implement a new base class called Instrumentation
.
You can find our tracking issue here: open-telemetry/opentelemetry-js#1074
I'm in the process of migrating the http/https plugin (i'll ping here with the PR once its done), there is already the graphql
plugin that have implemented: https://github.com/open-telemetry/opentelemetry-js-contrib/blob/master/plugins/node/opentelemetry-instrumentation-graphql/src/graphql.ts.
There was no discussion yet about removing the old plugin
mechanism as they still a lot of plugin to migrate but i will try to keep you updated on that
In the aws sdk instrumentation package, rpc
attributes are set from the normalizedRequest
.
opentelemetry-ext-js/packages/instrumentation-aws-sdk/src/utils.ts
Lines 40 to 47 in 243e19b
normalizedRequest
(v2 and v3) modify the request metadata values returned from the AWS SDK like this:
opentelemetry-ext-js/packages/instrumentation-aws-sdk/src/utils.ts
Lines 15 to 23 in 243e19b
Specifically:
service
name .toLowerCase()
operation
name .toCamelCase()
However, the specs for AWS SDK instrumentation give examples that contradict these patterns.
- ref: rpc.service
brief: "The name of the service to which a request is made, as returned by the AWS SDK."
examples:
- DynamoDB
- S3
- ref: rpc.method
brief: "The name of the operation corresponding to the request, as returned by the AWS SDK"
examples:
- GetItem
- PutItem
Based on the examples, we should return S3
instead of s3
and ListBuckets
instead of listBuckets
(for example).
This means using toUpperCase()
and a modified version of toCamelCase()
with the first character Upper Case.
looks like due to an open issue in yarn, running yarn install
/yarn remove
in subpackage, causes husky not to run, and for lint-staged hook, not to prettier
the new changes.
Original discussion here.
We should either find a workflow that works, or check prettier in the CI. Or we can just remove it altogether.
It appears all packages in this repo have been renamed from opentelemetry-plugin-XXXX
to opentelemetry-instrumentation-XXXX
in line with the upstream's change in terminology. It would be nice if the old plugin
packages could be deprecated in NPM in favor of their instrumentation
counterparts, since it doesn't look like they'll be maintained anymore. See the old HTTP plugin package for reference.
Fix the following issue -
`opentelemetry-instrumentation-aws-sdk/dist/src/types.d.ts:3:13 - error TS1192: Module '"/node_modules/aws-sdk/index"' has no default export.
3 import type AWS from 'aws-sdk';
~~~
Found 1 error.`
the solution -
import type { SQS } from 'aws-sdk'; ... ... export interface AwsSdkSqsProcessCustomAttributeFunction { (span: Span, message: SQS.Message): void; }
First off, I want to thank the maintainers of this repo for their hard work and fantastic contributions to the instrumentations in this repo and the OpenTelemetry project at large. I am from the AWS X-Ray team, and we have really enjoyed using many of the instrumentations here, particularly the AWS SDK one. We even officially recommend it for for AWS customers using OTel.
However as we bring our OpenTelemetry distro out of public preview and into GA, we are not able to rely on a third-party hosted package for AWS SDK instrumentation. We would like to offer assistance in maintaining it, and also hopefully exposure to an even larger customer base, while also allowing the Aspecto team to continue to have ownership of it. That is why we are proposing that the opentelemetry-instrumentation-aws-sdk
be moved upstream to the opentelemetry-js-contrib
repo. All current Aspecto maintainers would continue to be CODEOWNERS (or whatever equivalent we come up with for package ownership in contrib) in addition to some engineers from AWS.
As discussed in #93, I believe @blumamir is now an approver in contrib, and I would hope we could discuss adding another approver from this repo so you'd be able to continue to iterate quickly in contrib as well. Being in contrib and having the @opentelemetry
namespace would also hopefully lend well-deserved discoverability and credibility to the project. I will leave it up to the maintainers to decide where the other instrumentations in this repo should reside, but as the only one tied to a vendor the AWS SDK one seems like the odd one out.
When a consumer is summarily cancelled (perhaps due to a queue being deleted while a consumer is actively consuming messages), amqplib
sends the consumer a null
message to indicate that the queue has been cancelled. This is acknowledged by the type of the message
parameter in the callback. Nonetheless, the code immediately assumes that it cannot be null
by accessing .properties
. In the event of a cancellation, this results in a TypeError
from the instrumentation code. ?.
as on L261 should be used instead before passing the null
message on to the consumer.
For additional reference, see: https://github.com/squaremo/amqp.node/blob/1b327eba45df6f417e598715aa9010f76407fc29/CHANGELOG.md#breaking-changes
ConfirmedChannel is currently not patched and not instrumented by the amqplib
plugin.
We need to add support for that as well
Hello Aspecto people,
First of all thanks for the Neo4j opentelemetry plugin!
I'm testing it now and have the following issue...
{
"stacktrace": [
"TypeError: Cannot read property '_host' of undefined",
" at Object.getAttributesFromNeo4jSession ([redacted]/node_modules/opentelemetry-instrumentation-neo4j/src/utils.ts:14:6)",
" at Transaction.<anonymous> ([redacted]/node_modules/opentelemetry-instrumentation-neo4j/src/neo4j.ts:62:46)",
" at _callee3$ ([redacted]/node_modules/neo4j-graphql-js/dist/index.js:226:35)",
" at tryCatch ([redacted]/node_modules/regenerator-runtime/runtime.js:63:40)",
" at Generator.invoke [as _invoke] ([redacted]/node_modules/regenerator-runtime/runtime.js:293:22)",
" at Generator.next ([redacted]/node_modules/regenerator-runtime/runtime.js:118:21)",
" at asyncGeneratorStep ([redacted]/node_modules/@babel/runtime-corejs2/helpers/asyncToGenerator.js:5:24)",
" at _next ([redacted]/node_modules/@babel/runtime-corejs2/helpers/asyncToGenerator.js:27:9)",
" at [redacted]/node_modules/@babel/runtime-corejs2/helpers/asyncToGenerator.js:34:7",
" at new Promise (<anonymous>)"
]
}
I did a bit of digging around and make it work on my end since I need to test the whole flow
On _connectionProvider: RoutingConnectionProvider
I don't have _address
coming from the neo4j driver!
Instead, there is _seedRouter: ServerAddress
with the fields needed (_host
and _port
)
_seedRouter: ServerAddress {
_host: '[redacted].databases.neo4j.io',
_resolved: null,
_port: [redacted],
_hostPort: '[redacted].databases.neo4j.io:[redacted]',
_stringValue: '[redacted].databases.neo4j.io:[redacted]'
}
So I have patched it locally with the code below and it is working fine
const address = connectionHolder._connectionProvider._seedRouter;
It would be nice if there are null/undefined check guards and some generic default values for attributes so it doesn't fail or let it fail but more gracefully!
eg:
Node version: v14.15.4 (npm v6.14.10)
Neo4j driver version: 4.2.2
opentelemetry-instrumentation-neo4j version: 0.2.2
Neo4j DB version:
Version: | 4.2.0
Edition: | Enterprise
I'm using Neo4j Aura managed DB from Neo4j
p.s haven't tested with local or self-managed Neo4j DB
Kind Regards,
Bojanche S.
See socketio/socket.io-website#277
The docs don't say that and maybe the maintainers have a good reason here, but, conceptually, if disconnect
and disconnecting
are reserved, so is connection
.
Hello,
I've created two simple apps: producer and consumer and enabled autoinstrumentation. I am getting spans from those apps in Jaeger, but they are not connected. I added the debug logging to console and see that on the producer side opentelemetry-instrumentation doesn't add any headers. Is there any specific step required to enable context propagation?
Thanks
typeorm tests are creating a new db connection in each test. If something goes wrong (assertion fails, exception thrown), the test does not close this connection which causes the following tests to fail as well.
We need to move the logic into afterEach
hook so tests clean the environment after they are done for whatever reason.
As withSpan
was removed, the plugin will not work anymore.
open-telemetry/opentelemetry-js#1764 (review)
Please mention in Readme to use version 0.14 till a fix is released.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.