Comments (8)
However, I'm interested in a variant of this pipeline:
subscribeOn
can only macro-fuse a scalar or callable source which have the output fusion capability of ASYNC. Sometimes, operators have special paths such as this which leads to inconsistencies in the matrix. I probably should remove the output annotation from subscribeOn
.
Generally, subscribeOn
can't fuse because it can't negotiate the fusion mode between its downstream and upstream: the onSubscribe
gets called in one thread but the actual source Publisher
is called later on another thread. The fusion protocol mandates that when the onSubscribe
returns, the pair has to be agreeing on the mode.
subscribeOn and observeOn
Generally, they can't.
Also a related question, is the async fusion in subscribeOn currently implemented?
No and won't be.
Finally is there a place where @smaldini and @akarnokd discuss what is happening in this repo?
Since there are only the two of us, we chat in hangouts quite frequently. However, the fusion-related discussions happened mostly half a year ago; today, we don't discuss the theory behind it anymore and only do maintenance-related work based on feedback from Reactor-Core users.
I would be very interested in following along!
Unless you are already a master of Reactive programming and Reactive Streams, you'd have found it confusing and under-explained. I believe the best distilled version of what we accomplished is in my two blog post about it: part 1, part 2.
from reactive-streams-commons.
Ah the part I was missing was the requirement to negotiate synchronously in onSubscribe. Makes sense that subscribeOn cannot as it is switching threads.
I've read the blog posts you linked (as well as much of the posts in your blog - thank you for creating such a useful resource BTW). I'm certainly no master of Reactive streams although I'm working on it :P (my current work in this area is playing around with zero-heap allocation - i.e. only stack allocated - reactive streams in Rust).
In any case, I'll be following closely with interest both your work on RxJava 2.0 as well as the maintenance work here :)
from reactive-streams-commons.
Sorry for reopening again - didn't think it was worth opening a new issue for this. But I was looking through the history of subscribeOn and I noticed that at one point (90e6655) there was a non-eager variant of subscribeOn which delayed onSubscribe until it received it from upstream.
What was the rationale for removing this variant? From what I could tell this would solve the problem you described above (requiring to negotiate on the same threead) and lend itself to fusion. I also notice that RxJava also has a TODO to move to calling onSubscribe to subscribe time. I guess that was determined based on your work here?
Thanks again for being so helpful to date :)
from reactive-streams-commons.
From what I could tell this would solve the problem you described above (requiring to negotiate on the same threead) and lend itself to fusion.
It prevented cancelling a subscription in time, which was an undesired behavior in general so we dropped it and stayed with the classical subscribeOn
.
from reactive-streams-commons.
and lend itself to fusion
What would you like to fuse together?
RxJava also has a TODO to move to calling onSubscribe to subscribe time
I just posted a PR that fixes that.
from reactive-streams-commons.
I'm not interested in fusing an particular operator together TBH. The most interesting operators for fusion (to me at least) seem to be the thread switching ones (observeOn and subscribeOn).
Thanks for the info on cancellation. I did think that might be the reason. Good to get confirmation.
from reactive-streams-commons.
subscribeOn
and observeOn
are async-boundary operators and have to limit fusion to prevent user computation to escape. Sometimes you can replace subscribeOn
with observeOn
and get fusion (such as with direct sources as range, fromArray, fromIterable, just, fromCallable) but we don't automatically do that.
from reactive-streams-commons.
Yes precisely because they are required to limit fusion is why I find them interesting - most of the other operators have quite a straightforward reasons for supporting/not supporting fusion. Concurrent processes have always interested me to see how far they can be pushed. :)
from reactive-streams-commons.
Related Issues (20)
- Expand Macro-Fusion paths
- Expand Micro-Fusion ( Back-Fusion and ConditionalSubscriber )
- Add PatitionPublisher and PublisherPartition
- Review and Mark Unbounded Operators
- Add IndexedQueue contract to optimize draining paths
- Explore timed microbatch operators
- Explore Backpressurable#getCapacity optimizations
- Make GroupBy and future partitioning operator store-agnostic HOT 3
- Use shaded dependency on JCTools instead of copy and paste HOT 7
- Unbounded FlatMap with slow consumer HOT 1
- 101 Reactive Gems (working title) HOT 22
- Reactive puzzlers (working title) HOT 2
- Gem 22) discussion HOT 5
- Question: Using Reactor streams HOT 3
- Clarification on the Scheduler.Worker.schedule contract HOT 3
- Throttling on leading and trailing edge HOT 11
- Add PublisherIndexOf
- Add PublisherEvery HOT 1
- Expand Micro-Fusion (Front Fusion)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from reactive-streams-commons.