x-cubed / event-store-client Goto Github PK
View Code? Open in Web Editor NEWJS client library for connecting to Event Store over TCP/IP
License: The Unlicense
JS client library for connecting to Event Store over TCP/IP
License: The Unlicense
Awesome library. Thanks for building it. We are using TypeScript within our projects and identified that you are also providing a type definition file. Cool! Unfortunately there is a discrepancy between the package.json
which lives in the master
branch and the one that gets distributed by npm. The one in the master
references the type definition file correctly and the actual package on npm does not has a types
attribute at all. Would it be possible to publish a patch version 0.0.11
with the correct package.json
in it?
Edit: It also seems that the typings within the master
are old. Would it also be possible to update them?
(Disclaimer I'm not sure if this error is coming from event store client, from event store itself, or something completely different that has not yet been identified.)
We have an event handler that throws the following error when it runs for a sufficiently long time:
Error: write after end
at writeAfterEnd (_stream_writable.js:166:12)
at Socket.Writable.write (_stream_writable.js:217:5)
at Socket.write (net.js:650:40)
at Connection.sendMessage (/home/app/tableservice/node_modules/event-store-client/lib/connection.js:533:21)
at receiveMessage (/home/app/tableservice/node_modules/event-store-client/lib/connection.js:160:22)
at Socket.onData (/home/app/tableservice/node_modules/event-store-client/lib/connection.js:81:21)
at emitOne (events.js:96:13)
at Socket.emit (events.js:188:7)
at readableAddChunk (_stream_readable.js:172:18)
at Socket.Readable.push (_stream_readable.js:130:10)
at TCP.onread (net.js:542:20)
/home/app/tableservice/tableEventStore/index.js:404
throw err;
^
Error: write after end
at writeAfterEnd (_stream_writable.js:166:12)
at Socket.Writable.write (_stream_writable.js:217:5)
at Socket.write (net.js:650:40)
at Connection.sendMessage (/home/app/tableservice/node_modules/event-store-client/lib/connection.js:533:21)
at receiveMessage (/home/app/tableservice/node_modules/event-store-client/lib/connection.js:160:22)
at Socket.onData (/home/app/tableservice/node_modules/event-store-client/lib/connection.js:81:21)
at emitOne (events.js:96:13)
at Socket.emit (events.js:188:7)
at readableAddChunk (_stream_readable.js:172:18)
at Socket.Readable.push (_stream_readable.js:130:10)
at TCP.onread (net.js:542:20)
If I restart the service running this event handler, it picks up where it died just fine and continues processing events.
Something that might be relevant - the event handler that appears to be the source of this error is subscribed to one stream. When it picks up a certain class of event, it opens up a different stream to read detailed events relating to the subscribed stream.
Looks like its actively zero out metadata in the writeEvent method.
I was having difficulties on handling circuit breaker when i lost connection due to unforeseen event (e.g., cluster node is not accessible). I tried to use different circuit breaker libraries to catch the "onError" to return the fail attempt but with no luck.
Noticed that when consuming events from a stream built by a projection, with the "resolveLinkTos" option set to true, the deserializer appears to ignore what's in the link: { } structure entirely. This means that I can't get at the event numbers from the projection stream, just the ones from the stream that the link-resolved event comes from. So there's no way to track sequential event numbers in a projection stream if I also want the linked events resolved.
Relevant code: lib/connection.js - unpackResolvedEvent(resolvedEvent) -> return unpackEventRecord(resolvedEvent.event);
See Google Groups issue I raised: https://groups.google.com/forum/#!topic/event-store/MIsb87kjvzE
The onData and receiveMessage functions in connection.js need to use streams to support large messages.
messageLength: 4294967299
UNCAUGHT EXCEPTION!!!!!
[RangeError: Invalid typed array length]
RangeError: Invalid typed array length
at new Uint8Array (native)
at createBuffer (buffer.js:23:15)
at allocate (buffer.js:98:12)
at new Buffer (buffer.js:53:12)
at Socket.onData (/Users/Corey/GitHub/UpdecSync/node_modules/event-store-client/lib/connection.js:92:47)
at emitOne (events.js:77:13)
at Socket.emit (events.js:169:7)
at readableAddChunk (_stream_readable.js:153:18)
at Socket.Readable.push (_stream_readable.js:111:10)
at TCP.onread (net.js:531:20)
Hi @x-cubed,
I wonder if you can provide some insight into making sure the each event is processed before moving to the next one.
( async event => { await eventHandler.handle(event); }),
but it seems to ignore that as I see other events being handled before the previous one finished.
In the type definition:
subscribeToStreamFrom(streamId: string, fromEventNumber: number, ...)
Should be:
subscribeToStreamFrom(streamId: string, fromEventNumber: number | null, ...)
It is missing the | null
, for the fromEventNumber
parameter. Without it, you cannot read from the beginning of the stream.
Hi,
I've been investigating issue where my js code intermittenly stop pushing event to eventstore. Once I put some logging i found these:
[PID:01424:012 2016.08.13 15:07:30.341 TRACE TcpConnectionManager] Closing connection 'external-normal' [127.0.0.1:61047, L127.0.0.1:1113, {7c3c6a87-ec80-404c-b0f5-916f7ef81e72}] cleanly. Reason: HEARTBEAT TIMEOUT at msgNum 3742004
[PID:01424:012 2016.08.13 15:07:30.341 INFO TcpConnection ] ES TcpConnection closed [15:07:30.341: N127.0.0.1:61047, L127.0.0.1:1113, {7c3c6a87-ec80-404c-b0f5-916f7ef81e72}]:Received bytes: 284729333, Sent bytes: 6448674876
[PID:01424:012 2016.08.13 15:07:30.341 INFO TcpConnection ] ES TcpConnection closed [15:07:30.341: N127.0.0.1:61047, L127.0.0.1:1113, {7c3c6a87-ec80-404c-b0f5-916f7ef81e72}]:Send calls: 3741924, callbacks: 3741924
[PID:01424:012 2016.08.13 15:07:30.341 INFO TcpConnection ] ES TcpConnection closed [15:07:30.341: N127.0.0.1:61047, L127.0.0.1:1113, {7c3c6a87-ec80-404c-b0f5-916f7ef81e72}]:Receive calls: 3742005, callbacks: 3742004
[PID:01424:012 2016.08.13 15:07:30.341 INFO TcpConnection ] ES TcpConnection closed [15:07:30.341: N127.0.0.1:61047, L127.0.0.1:1113, {7c3c6a87-ec80-404c-b0f5-916f7ef81e72}]:Close reason: [Success] HEARTBEAT TIMEOUT at msgNum 3742004
I found in .NET client library have the functionality:
connectionSettingsBuilder.KeepReconnecting();
connectionSettingsBuilder.KeepRetrying();
Is there a workaround for this issue ? or should I reconnect when onError / onClosed event called ?
After a few hours I always get the error
Error: This socket has been ended by the other party
at Socket.writeAfterFIN [as write] (net.js:278:12)
at Connection.sendMessage (/.../node_modules/event-store-client/lib/connection.js:504:21)
at Connection.writeEvents (/.../node_modules/event-store-client/lib/connection.js:423:10)
at /.../eventstore.js:43:18
My code (the line 43 from the stacktrace is the one with eventStore.writeEvents
):
let connected = false;
const eventStore = new EventStore.Connection({
host: config.eventStore.host,
port: config.eventStore.port,
debug: config.eventStore.debug,
onError: (error) => {
connected = false;
logger.error(error);
},
onConnect: () => {
logger.info('connected to eventstore');
connected = true;
},
});
function emitToEventStore(message) {
if (!connected) {
return Promise.reject(new Error('EventStore not connected'));
}
return Promise.resolve()
.then(() => {
const event = {
...
};
return new Promise((resolve, reject) => {
eventStore.writeEvents('streamname', EventStore.ExpectedVersion.Any, true, [event], null, (result) => {
if (result.message) {
return reject(new Error(result.message, result));
}
return resolve();
});
});
})
.then(() => logger.info(`posted message # ${message.properties.messageId} to eventstore`))
.catch((error) => {
logger.error(`could not post message # ${message.properties.messageId}`, error);
});
}
Is there any way to react to a disconnect?
We are getting subscriptions dropping intermittently using this library. Although we log errors on both connection drop and on Error, the subscription just drops silently, not giving any hint as to why it has stopped receiving events. This doesn't appear to be after any set period of time, sometimes it is almost straight away, sometimes after a few hours. With DEBUG mode on, we can see the heardbeats continue on without fail after the last event before the drop.
Any idea what might be going on and how to debug this further?
How does one start a catchup subscription through this client?
Do you plan to implement transaction scope?
Since the catchup subscription is not currently supported, I assume that competing consumer subscriptions are also not supported?
I'm looking to add this functionality. Are there any technical reasons why this hasn't been done? Or is it simply a missing feature?
Hi, we love and use your library! One thing I can't figure out though is how to detect a connection error.
We use writeEvents
and wait for the success-callback. If the event store went down for some reason I can see a log output of the connection error, but I don't see a way to catch and react to it in my code. Do you have a hint for me?
Hi, I am using this library to interface with the event store 5.0.8.
I am able to write to the streams and by checking in the web interface I can see the contents correctly, such as the event type and the data.
But when I subscribe, I see that some fields are returned as Buffer. Example of a console.log of one event retrieved shows
{
streamId: '$ce-mystream',
eventNumber: 54,
eventId: 'bad6ce85-edff-4c13-a6fa-63c7f755257a',
eventType: '$>',
isJson: false,
created: 2020-05-10T19:40:23.166Z,
data: <Buffer ...
I could not find how to have access to the original value
The scenario was ... (am migrating crud sql to aggregates)
using one connection
Maybe 35 rows of data that - per row - converts into 8 calls into an aggregate root. This in turn would per call load 1-2 aggregates (no more than 8 events per stream) by reading stream forwards and then potentially write an event to each of those streams.
If i limit the rows of data to about 24 it all works - at this point i have put nothing in to monitor the number of concurrent read stream forwards i am doing.
If i operate on the rows sequentially rather than a ton of promises in parrallel it all works.
The problem is not the failure - its the lack of feedback.
When i fails i do get onClose(false) getting called at the connection level - but nothing that causes each aggregate to fail. The callback simply isn't made as far as i can tell.
Still investigating. Posted this incase someone can point out how i'm being silly!
Otherwise i guess a solution would be to end up adding some kind of timeout into the read forward subscription chunks?
I may be missing something here, but I can't find a way to get the most recent event in a stream without doing two calls.
I would have thought readStreamEventsBackwards with a max number of 1 would do it, but it requires a "fromEventNumber" which prevents me from getting the last one as I don't know the last event number prior to the query. I'm not sure why "fromEventNumber" is a required field on this query... what am I missing?
Hello,
I've tried to use this library to connect angular 5 clients to an eventstore backend, unfortunately I'm running into multiple problems. I've installed the application using npm install event-store-client
and inserted import * as EventStoreClient from "event-store-client";
at the top of test-es.ts - the TS file responsible for connecting to the EventStore server (based on the provided example.js file). After building the app with ng serve
, I first had a first runtime error when loading the page that calls to test-es.ts:
get http://localhost:4200/ClientMessageDtos.proto error 404 (not found)
I solved this by manually deploying the proto file in .angular-cli.json
Then I run into another runtime error:
ERROR TypeError: net.connect is not a function
at new Connection (connection.js:49)
Therefore I was wondering if anyone had successfully managed to use this library in an Angular+typescript client-side application.
And btw, thanks for this very interesting library !
Hello
Does this client support an SSL connection to Eventstore?
Thanks
Maarten
I'm curious, @x-cubed. Is there a wishlist for version 1.0.0? Can you post it? My team is beginning to build Node services using this client, and if we have an opportunity to further contribute (and if we find time to do so) we can continue to help.
Cannot find facilities for subscription to all events being written to EventStore. In C# official client we have SubscribeToAllFrom(lastCheckpoint, ...) for such purposes.
Are there any alternatives in event-store-client?
Hi,
There seems to be a small bug in the Typescript definition file for the createGuid method.
I believe this needs to be marked static: like this:
/***
* Helper function to create a new v4 UUID to use for event IDs or correlation IDs
*/
static createGuid(): Buffer;
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.