Comments (19)
I may be missing something obvious, but I don't understand why there would be Service::poll_outstanding
, since call
already returns a Future
to poll to make progress.
from tower.
@jonhoo For clarity, could you move finish
and enqueue
to a ServiceExt
trait?
I'm also leaning towards not having any default implementations for functions. I think that the main implementors of Service
will be middleware and client / server implementors. In those cases, they will want to be sure to add implementations of all functions, if only to proxy to the inner service.
For the simple case, we can defer to service_fn
:
let my_service = service_fn(|request| {
// do something with request
Ok(response);
});
from tower.
In theory, I really want something like this as it would reduce the required overhead of using the Tower Service abstraction.
That said, I thought more about it, and it is a bit more complicated...
The "drive" function would have to do two things:
- Drive the internal state of the
Service
- Notify the caller when it is safe to drop the
Service
(i.e., the socket is closed and all response futures have completed).
The second point also implies that there needs to be some way to tell the service to start shutting down.
So, at the very least, there would need to be two additional functions:
- Some "drive" function.
- Some function to signal the service should start shutting down.
(Note that Sink
provides APIs to achieve this).
This adds non trivial complexity to using a Service
. It would be unfortunate if all users of Service
are subject to this additional API.
This implies that there would need to be a separate trait. How would that work?
from tower.
I don't really have an answer, just more observations about why the current situation isn't ideal: if you do work to "drive" the returned future(s) in poll_ready
, you now end up in the weird situation where you need to basically alternate between polling the future and calling poll_ready
until the former resolves. That's a pretty awkward pattern (maybe you have a nice way to deal with it?). It could be that all tower
needs to provide is some nice way to say "drive the Service
until this future resolves`?
from tower.
The other advantage to have Service: Future
(at least in some cases) is that I could stick a bunch of them in FuturesUnordered
or something like that. In my case, I have multiple tokio_tower::pipeline::Client
, one to each "shard" of a database. I send a request on some subset of them, and now I need to drive them all until I the reply futures resolve...
from tower.
If you stash them in FuturesUnordered how do you call them?
from tower.
FuturesUnordered::iter_mut
, but it's obviously not ideal. FuturesOrdered
may be needed to give predictable access to the services.
from tower.
Bah, FuturesOrdered
also doesn't allow lookup :'(
from tower.
it would be possible to impl a similar structure that does enable lookup though...
from tower.
I actually think modelling Service
as a Sink
makes a tonne of sense. I don't think it should actually be Sink
though, because Sink
doesn't have a notion of returning a future from a send (afaict). That gives us both shutdown and polling! We could even then add a .drive()
method (similar to .ready()
) that keeps polling until there's no more work for the Service
to do (i.e., no outstanding requests).
from tower.
For a more concrete example of where this applies, see: https://github.com/jonhoo/tokio-tower-wip/blob/861f273abea1682d1be90959c34a54d83f291b86/tests/pipeline/client.rs#L40
from tower.
Some further discussion happened on Gitter. A brief-ish summary:
- We'd like implementors of
Service
not to have to always insert anmpsc
of some sort and thenspawn
something onto the current executor. - That means there has to be a way to "drive" the
Service
to make progress on in-flight requests. Service
wrappers also need a way to communicate to aService
that there are no more requests coming, and that it should shut down once idle.- We'd like to avoid adding complexity to simple
Service
implementations.
It seems like we need to figure out what methods Service
needs, and a contract for when what should be called. We could add an additional extension trait of some sort for "fancy" Service
impls, but it seems easier to just provide sensible default fn
s. I propose:
trait Service {
/// Called to make room for another request.
///
/// Should call `poll_outstanding` as necessary to finish in-flight requests.
///
/// The default implementation always returns `Ready`, which must be safe because `call`
/// (per spec) must always be prepared to be called, even if not ready.
///
/// Implementors are encouraged to implement this if `call` can ever fail.
default fn poll_ready(&mut self) -> Result<Async<()>, Self::Error> {
Ok(Async::Ready(())
}
/// Called to make progress on in-flight requests.
///
/// Should return `Ready` when all outstanding requests have been serviced.
fn poll_outstanding(&mut self) -> Result<Async<()>, Self::Error>;
/// Called when there will be no more calls to `call` or `on_ready`.
///
/// This method will be called at most once.
/// The default implementation does nothing.
default fn on_shutdown(&mut self) { }
/// Called when there will be no more calls to `call`.
///
/// The returned `Future` resolves when the shutdown has been completed,
/// and all outstanding requests have been responded to (i.e., when
/// `poll_outstanding` next returns `Ready`.
///
/// This will implicitly call `on_shutdown` on the `Service` before polling
/// outstanding requests.
fn shutdown(self) -> ShutdownFuture<Self> { ... }
/// ...
fn call(&mut self, req: Self::Request) -> Self::Future;
/// Returns a `Future` that resolves when the given future resolves, and
/// internally calls [`poll_outstanding`] to drive this `Service` forward.
///
/// Akin to `future::select(fut, self)`.
fn finish<F>(self, fut: F) -> OutstandingPoller<Self, F> { ... }
/// Returns a `Future` that resolves the first time `call(req)` completes
/// after `poll_ready` has returned `Ready`. It produces `Self` and the
/// `Future` returned from `call`.
fn enqueue(self, req: Self::Request) -> ReadyCallPoller<Self> { ... }
}
from tower.
Following some further discussion on making the trait a bit more like Sink
, and relying more on callers to maintain contract, how about:
trait Service {
/// Called to make room for another request.
///
/// Should call `poll_outstanding` as necessary to finish in-flight requests.
///
/// The default implementation always returns `Ready`, which must be safe because `call`
/// (per spec) must always be prepared to be called, even if not ready.
///
/// Implementors are encouraged to implement this if `call` can ever fail.
default fn poll_ready(&mut self) -> Result<Async<()>, Self::Error> {
Ok(Async::Ready(())
}
/// Called to make progress on in-flight requests.
///
/// Should return `Ready` when all outstanding requests have been serviced.
fn poll_outstanding(&mut self) -> Result<Async<()>, Self::Error>;
/// Called after there will be no more calls to `call`.
///
/// `poll_close` should ensure that all in-progress requests resolve, as well
/// as perform and finish any required service cleanup.
default fn poll_close(&mut self) -> Result<Async<()>, Self::Error> { self.poll_outstanding() }
/// ...
fn call(&mut self, req: Self::Request) -> Self::Future;
}
trait ServiceExt: Service {
/// Returns a `Future` that resolves when the given future resolves, and
/// internally calls [`poll_outstanding`] to drive this `Service` forward.
///
/// Akin to `future::select(fut, self)`.
fn finish<F>(self, fut: F) -> OutstandingPoller<Self, F> { ... }
/// Returns a `Future` that resolves the first time `call(req)` completes
/// after `poll_ready` has returned `Ready`. It produces `Self` and the
/// `Future` returned from `call`.
fn enqueue(self, req: Self::Request) -> ReadyCallPoller<Self> { ... }
}
from tower.
@seanmonstar the observation is that for some Service
implementations, there may be a resource inside the Service
that has to be polled for the returned future to make progress. For example imagine the Service
wraps a TcpStream
that will eventually yield something that resolves the future. The Service
has to continue to be polled in some way, otherwise the Future
returned from call
will never resolve.
from tower.
@seanmonstar
Sometimes, you can't drive the shared service state from each future. In this case, today, you need to add a message passing layer.
Also, there are cases where the service needs to be driven when there are no futures (think h2's ping / pong).
from tower.
Also, we may want to bikeshed poll_outstanding
as a name as the intent is to allow the service to do work even if there are no outstanding responses. For example, HTTP/2.0 needs to be able to respond to Ping frames.
from tower.
We could just call it poll
. The reason I went for poll_outstanding
was to indicate that even if it returns Ready
, that does not mean that the Service
is "done". It just means that any futures that have been given out have been resolved.
from tower.
Much more discussion on Gitter today, culminating in this question (all chat history edited for clarity):
@carllerche: There is also an extra wrinkle… Say you want a “spawned” service so that you don’t have to worry about calling
poll_outstanding
,poll_close
... How do you get that without double spawning if you takeT: Service
?
@jonhoo: Hmm, yeah, it really does feel like the "inner"Service
is different. Basically,Buffer
changes the contract of theService
. I almost wonder if we'll tons of things basically relying on aBuffer
around whateverService
they operate on, or that they all explicitly add aBuffer
"to be safe", and we end up withBuffer<Buffer<Buffer<S: Service>>>
, which would be very sad indeed.
It could be that we want aService
trait with justcall
(and maaaaaybepoll_ready
) like today which is what all the middleware operate on and expose, and then aDirectService
which has all thepoll_*
, and which you can wrap with aBuffer
if you want it to be spawned. Or you could directly implementService
and spawn internally if you really wanted to.
@carllerche: It might be worth experimenting w/ keeping a separate trait basically and see how it all fits together. Having two traits wasn’t super successful withReadyService
, but maybe this would be different. In this case,T: Service
wouldimpl DirectService
and you would doDirectService::buffer(…) -> Service
.
Note also that we should not have DirectService: Service
; quoth @carllerche:
The issue is, if a function takes
T: Service
, andDirectService: Service
, then you could pass aDirectService
to that thing and it would be broken.
There's also the worry that we now need to have lots of things be able to take both DirectService
and Service
:
@carllerche:
tokio-tower
’sServer
will need two separate constructors: one from normal service, one fromDirectService
(unfortunately)
@jonhoo: I guess what I mean is that most middleware will probably just operate onService
@carllerche: Most middleware intower
can probably take both
@jonhoo: Probably, though that means they will also themselves implementDirectService
, notService
@carllerche: They will have impls based on the inner type
@jonhoo: So I guess the idea is that people construct a bigDirectService
hierarchy, and then do::buffer
at the end. Which maybe is what you want?
@carllerche: Yes. Another problem is, middleware constructs will not be able to have awhere
bound unless there isfn new(T: Service)
andfn new(T: DirectService)
.
@jonhoo: I do think we'll want to be able to provide aService
to something that takes aDirectService
. That'd make it a lot more ergonomic
@carllerche: Specialization would make our lives much better
It seems like we're basically agreed that DirectService
is the thing to add. With one caveat (quoth @carllerche):
IMO,
DirectService
should not leak into “main” tower until it is proven out.
Well, it can be in the tower crate, butService
should be considered the main way to do things.
So, we would have:
trait Service<Request> {
/// Like today's Service::poll_ready.
/// Should this even be present on `Service`, or just on `DirectService`?
fn poll_ready(&mut self) -> Result<Async<()>, Self::Error>;
/// Like today's Service::call
fn call(&mut self, req: Request) -> Self::Future;
}
trait ServiceExt<Request>: Service<Request> {
/// Returns a `Future` that resolves the first time `call(req)` completes
/// after `poll_ready` has returned `Ready`. It produces `Self` and the
/// `Future` returned from `call`.
fn enqueue(self, req: Request) -> ReadyCallPoller<Self> { ... }
}
trait DirectService<Request> {
/// Called to make room for another request.
///
/// Should call `poll_outstanding` as necessary to finish in-flight requests.
fn poll_ready(&mut self) -> Result<Async<()>, Self::Error>;
/// Called to make progress on in-flight requests.
///
/// Should return `Ready` when all outstanding requests have been serviced.
fn poll_outstanding(&mut self) -> Result<Async<()>, Self::Error>;
/// Called after there will be no more calls to `call`.
///
/// `poll_close` should ensure that all in-progress requests resolve, as well
/// as perform and finish any required service cleanup.
default fn poll_close(&mut self) -> Result<Async<()>, Self::Error> { self.poll_outstanding() }
/// Like today's Service::call, but with the caveat that poll_outstanding must
/// continue to be called for the returned futures to resolve.
fn call(&mut self, req: Request) -> Self::Future;
}
- various
type
associated types.
We may also want an (internal) struct
that wraps a Service
and impl DirectService
with empty implementations for all fn
s so that it's easy to write middleware or servers that are able to drive an inner DirectService
, and which can also trivially then accept a "regular" Service
.
from tower.
We'll probably also want to be very clear in the docs for Service
and DirectService
how they.re different. In particular, DirectService
's contract is that returned futures will only complete if poll_outstanding
is called. Service
is a DirectService
that is implicitly driven, so consumers only need to poll the future returned from call
.
from tower.
Related Issues (20)
- Adding Service Wrapper that allows to share services for compatibility with Axum route layers HOT 4
- Support weights for request rate limiting HOT 1
- Can I ask one question? HOT 4
- Publish `0.5` release HOT 5
- `Retry<RetryPolicy, RateLimit<Client>>` does not work HOT 7
- Idea: Preventing Inappropriate Service Invocation HOT 1
- Returning a response from a tower layer HOT 2
- AsyncFilterLayer is missing Clone impl
- Have MakeBalance and MakeBalanceLayer example?
- `Reconnect::new()` Generic parameters are redundant HOT 2
- Adding option_layer causes trait bound unsastisfied HOT 1
- Publish release without pin-project (with pin-project-lite) HOT 5
- `tower::service_fn` docs don't say that you need the `util` flag HOT 3
- MQTT client adapter / framework HOT 1
- “Fan out” services? HOT 3
- Consider using `ControlFlow` for retry `Policy`.
- Extending `Building a middleware from scratch` guide
- breaking change in tower design (0.6 or beyond): first class support for async fn traits HOT 20
- unexpected behaviour of `RateLimit`
- experiment with permit based service framework HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tower.