RawMessage and AdaptedMessage about eventstore-tcp

[Kerollmops]

I have a little questionL Do you think this is a right thing to let the user make it's own WriteEventResponse for example ?

Is there response types that needs to be construct by another way than the easy one ?

For example response messages cannot be build, are locked.

[koivunej]

I first started out by providing a rustic api only. After a while I realized I do know how all these values in the protobuf types are being used, and ran out of steam while trying to figure things out. Decided to cut my losses by moving the "rustic" stuff over as adapted and keep it separated from the raw which is the minimal adaptation over the protocol.

This is too much to handle, too much errors and wheels. The classic ProtobufType -> EventStoreMessage can be a great improvement to this lib'.

I think this would be the eventstore_tcp::raw::RawMessage enum which you can access using the crate level example (just remove the try_adapt().unwrap()):

fn main() {
    // initialization
    let value = client.and_then(|client| {
        /* build a request, send it */
    }).and_then(|resp| {
        match resp.message.try_adapt().unwrap() {
                   ^^^^^^^
                      |
             the RawMessage value you can use to get what you are looking for

My idea with adapted was to map the protocol data types into idiomatic rust values, whch would be easier to use, like adapted::AdaptedMessage::WriteEventsCompleted(Result<WriteEventsCompleted, WriteEventsFailure>) instead of the protocol level equivalent.

Do you think this is a right thing to let the user make it's own WriteEventResponse for example ?

I am not sure what you mean. There is no such type in the project?

For example response messages cannot be build, are locked.

I do not think so. I tried putting this in tests/test.rs:

extern crate eventstore_tcp;
use eventstore_tcp::raw::client_messages;

#[test]
fn make_response() {
    let _ = client_messages::ReadStreamEventsCompleted {
        events: vec![],
        result: None,
        next_event_number: -1,
        last_event_number: -1,
        is_end_of_stream: false,
        last_commit_position: -1,
        error: None,
    };
}

I think the fact that the protocol level allows such message like the above ReadStreamEventsCompleted is good argument for providing something like adapted. Not that I'm saying it's any fault of the protocol description.

[Kerollmops]

I agree about the way you want to hide raw data to the user but the way you make it seems to be made backwardly. You force the user to call try_adapt followed by an error handling (unwrap here).

I think the message needs to be rustic and, why not, let the user read the raw data. (I don't see a great reason to let it do that...).

When I said:

Do you think this is a right thing to let the user make it's own WriteEventResponse for example ?

I make a reference to the WriteEventsCompleted. This is accessible to the user but, for me, this is not a great thing, this is to useless/dangerous. Furthermore this message cannot be send by the client. (Except if the client is set as a cluster or something like that, I don't know).

I am working on a way to completely hide RawMessages to the user.

[Kerollmops]

I don't know if there is a way to put a futures closure to a given response.
For example, I create a NewEvent, send it to the GetEventStore database and specify a callback when a response for this event is sent.

The big thing is to let the user see a Result<NewEventSaved, NewEventError> (or something like that) as argument of this futures closure. He can make it's code really clear to given events without the ugly way to deal with the Adapted/RawMessage enum.

We need to manage the RequestId (UUID) for the user, don't let the user do the hard stuff.

client.and_then(|client| {
    /* build a NewEvent request, send it */
}).and_then(|resp| {
    // here we receive a Result<NewEventSaved, NewEventError>
    // not a big enum of all message types
    match resp {
        Ok(x) => /* this is good */,
        Err(e) => /* this is not good */
    }
});

To achieve this can we store the closure somewhere in the ClientProto, with the UUID corresponding ? I read somewhere that there is a ClientProto clone for each request, is that true ?

[koivunej]

I think the message needs to be rustic and, why not, let the user read the raw data. (I don't see a great reason to let it do that...).

One good reason is that abstractions always leak and it can be difficult to find good abstractions at first sight. I think you can build good abstractions on the current raw package access. You can use the raw access when the abstraction fails.

For example, I create a NewEvent, send it to the GetEventStore database and specify a callback when a response for this event is sent.

This is exactly what happens in the crate level example but yeah, with the builder usage and you need to match the raw message, or the adapted message. I think all you need is to provide conversions from NewEvent and conversion from the response package to your NewEventError and that's it. (Though for namings sake I think it'd best to keep aligned to the command names used in the protocol -- WriteEvents meaning writing 1..n events.)

I have been thinking of this sort of API ("nice api" in the README), but thought it'd be best to get the basic functionality like reconnection and subscriptions working first. This would require building some sort of stable struct that would hide the connection management and perhaps spin up another thread for running the tokio_core::reactor::Core. There is minidb which I found recently that is an example of this kind of facade.

We need to manage the RequestId (UUID) for the user, don't let the user do the hard stuff.

Yeah, the package is protocol level stuff only. There is actually no management required by the user at the moment, but since it's called correlation_id I thought that maybe it'd might be useful to correlate it with something, like to the process generating the events. Perhaps it's best to leave it only between the client and the server, not sure.

To achieve this can we store the closure somewhere in the ClientProto, with the UUID corresponding ? I read somewhere that there is a ClientProto clone for each request, is that true ?

You can see it for youself in src/client.rs but no, I don't think it's for each request, it is for each connection. It is the description of the protocol from clients perspective. UUIDs are used as you can have multiple ongoing requests to the server at the same time. You do not need to provide it for the builders at the moment either, you can use None as in the example I've now linked a few times.

[koivunej]

Been thinking the nice api a bit more. I think that a nice api could be an infallible conversion from the adapted messages, but without them, the conversion from raw would always have the ability to fail. So, I don't think that those should be gotten rid of until a better idea comes along. Perhaps the most useful right now with respect to code quality would be to have better macros for the conversions.

[koivunej]

Yeah this is still a mess. I still feel it was a good idea to pursue without re-reading all my writing above but looking at the code right now perhaps it was a bit difficult approach.