XHR option to trigger busy indicators about fetch

[josh]

Also consider that the "Stop" button is closely linked to "in progress".
It'd be kinda neat if that put the user in control of canceling the request
if they wish. Shout to cancelable fetch promises.
On Mon, Jan 19, 2015 at 1:22 AM Guillermo Rauch [email protected]
wrote:

It depends on how it's implemented. The iOS status bar binary loading
state works pretty well, for example.

—
Reply to this email directly or view it on GitHub
#19 (comment).

[gaearon]

@rauchg

I was thinking about bad libraries messing with a boolean field that you have no control over. Imagine some SDK doing that. Maybe it's a library's problem though.. I'm not sure it's good to encourage bad patterns by exposing API that's too easy to misuse.

And for a good API, I can't think of a better fit than promise. It represents an asynchronous operation in the language. You don't need to use promises in your code to represent async operations if you don't want to, but wrapping any async result (socket, XHR, whatever) in a promise is as straightforward as any other async API could be. If you want to track async operations in the UI, you need a way to represent an async value, and promises are baked into the language precisely to be the way to do that.

If core browser APIs expose promise-based async operations such as fetch, IMO it's weird for async operation tracker to expose lower or higher level API. Otherwise, why make fetch promise-based?

Finally, I can't build promise-based layer on top of isBusy because I have no guarantees somebody (including third-party libraries) doesn't set isBusy directly, thus making my layer completely unreliable.

[annevk]

Is there implementer interest to the extent they are willing to put UX resources on this? (Opera used to show all XMLHttpRequest activity and it was very bad (due to the background resources you mention).)

[josh]

I would ❤️ this so much and we'd totally use this on @github for pretty much all our page transitions instead of custom spinners.

[jussi-kalliokoski]

👍, but instead I'd suggest programmatic control over the feature, because loading may include JS, fonts, images or processing in a worker (e.g. if a data blob is cached, but requires processing before being fed to a view).

[jussi-kalliokoski]

And also because the actions may be canceled (e.g. a user clicks on a wrong link and then the correct one).

[d2s]

More consistent UI behaviour between websites (and different browsers…) could make things much more clear for the users. Also, less code to implement for front-end developers, reducing need for additional building blocks.

[rauchg]

This should not be tied into XMLHttpRequest, because of possible alternative transports in play (WebSocket). Also because a sequence of requests might be needed to trigger the following page.

Maybe we should consider something on the navigator or window object.

navigator.setBusy(true);

I personally would like to see this happen not just because of the perception of slowness, but because of the duplication of user-agent UI that's happening with the "slim progress bar" trend (as seen on YouTube, Medium, etc)

The performance angle is more nuanced. Many single page applications solve it by making the transition to the new page immediate, and displaying placeholders there (eg: Facebook Newsfeed)

[josh]

It's more than a binary state, you really want something associated with
progress events.

I kinda doubt UAs are going to give complete direct access to something
like navigator.setProgress(0.75) given the ability for abuse, even though
it'd be pretty cool.

Restricting to something request-like would cover most use cases.
On Mon, Jan 19, 2015 at 12:33 AM Guillermo Rauch [email protected]
wrote:

This should not be tied into XMLHttpRequest, because of possible
alternative transports in play (WebSocket). Also because a sequence of
requests might be needed to trigger the following page.

Maybe we should consider something on the navigator or window object.

navigator.setBusy(true);

I personally would like to see this happen not just because of the
perception of slowness, but because of the duplication of user-agent UI
that's happening with the "slim progress bar
https://github.com/rstacruz/nprogress" trend (as seen on YouTube,
Medium, etc)

The performance angle is more nuanced. Many single page applications solve
it by making the transition to the new page immediate
http://rauchg.com/2014/7-principles-of-rich-web-applications/#act-immediately-on-user-input,
and displaying placeholders there (eg: Facebook Newsfeed)

—
Reply to this email directly or view it on GitHub
#19 (comment).

[rauchg]

It depends on how it's implemented. The iOS status bar binary loading state works pretty well, for example.

[gaearon]

navigator.setBusy(true);

Different parts of code may want to push/pop an async operation so this wouldn't work well as the primary user-facing API IMO.

Instead I'd rather prefer something like this:

window.attachToBusyIndicator(promise);

Internally, we'd keep track of promises in progress and only show spinner when there is >= 1 such promise. When promises resolve, we remove them from the queue.

var busyPromises = [];

function refresh() {
  NativeImplementation.showBusyIndicator = busyPromises.length > 0;
}

window.attachToBusyIndicator = function (promise) {
  function detach() {
    busyPromises.splice(busyPromises.indexOf(promise), 1);
    refresh();
  }

  promise.then(detach, detach);
  busyPromises.push(promise);

  refresh();
}

This also works nice with cancellation because, whatever approach you choose for cancelling promises, if cancellation implies rejection (and it should), our counter will decrement on cancellation. Example of a nice cancellation API.

[fabiosantoscode]

I completely agree that the browser's busy indicator should not be tied to XHR or fetch(). What about JSONP, WebSockets, PeerConnections? they are perfectly valid ways of getting data which you have to wait on.

Busy indicators, progress indicators, are visual indicators, and thus in principle should not be triggered by network operations, even as an opt-in.

[rauchg]

@gaearon I actually prefer browser APIs to be minimal. You can add the Promise layer on top of it trivially. Just like not everyone uses XHR exclusively (the very premise of my post), not everyone has to use Promise to trigger the indicator.

[rauchg]

@gaearon

Your layer can be implemented by listening to busyChange events and querying an isBusy flag.
That said, I agree that Promise could increase harmony between 3rd party libraries.

[gaearon]

@rauchg

Your layer can be implemented by listening to busyChange events and querying an isBusy flag.

Say I have two pending promises in my layer. Thus I set isBusy = true.

Then nasty module X comes along and does .isBusy = true.
But it's busy already. Would the busychange event fire?

If it's a change event, it wouldn't, but then, when my pending promises have resolved, I'd have no way to know from my layer that somebody else wanted to keep the app busy.

If we're sure we don't want a Promise API for that, I still suggest two methods over isBusy:

window.setBusyIndicator(): operationId
window.clearBusyIndicator(operationId)

These can be called safely from different modules.

[rauchg]

I'd have no way to know from my layer that somebody else wanted to keep the app busy.

It's the responsibility of "nasty module X" to retain its busy state for as long as it considers that it's busy, by watching for busyChange events and resetting it to true if needed.

[rauchg]

That said, to avoid contention issues, operationId could be nice.

[gaearon]

It's the responsibility of "nasty module X" to retain its busy state for as long as it considers that it's busy, by watching for busyChange events and resetting it to true if needed.

But then the order in which modules subscribed to busyChange would potentially affect the result. IMO it's a recipe for trouble.

[bloodyowl]

I'd go for something like this, giving control in user-land :

var busy = new navigator.BusyIndicator()

busy.start() // starts the busy indicator without progress
busy.end() // stops the busy indicator

busy.setProgress(.3) // sets the busy indicator at 30%, and starts it if it's not already
busy.setProgress(1) // calls (`busy.end`) 

fetch("some/uri")
  .then(busy.end)

this way we'd be able to :

• have a "busy indicator" instance for different parts of the UI
• use busy indicators in multiple places, and compute setProgress values in the navigator-land
  to have a "global" busy indicator state

[gaearon]

I'd leave progressing out of this.

Otherwise it's not clear how progress value should be calculated from several (possibly partly unspecified) task progress values.

[bloodyowl]

yes that would be a difficulty, though could be interesting from a UX point of view

[rauchg]

Most browsers don't show progress bars for page transitions, so I'd be ok with just a binary busy state.

[domenic]

All this talk is way premature given that we haven't gotten interest in this feature from even a single implementer yet :)

[Lewiscowles1986]

@domenic do you mean no interest from browser vendors or client-side fill's?

Client-side fill's for the functionality visibly to the user are easy, it's the browser support for showing on tab etc that will be sketchy

[domenic]

I mean browser vendors, the ones implementing this spec :)

[duanyao]

I think this problem can be solved by add a new API: global fetch events.

These events should be fired when the page is fetching any resources, no matter caused by XHR, fetch, <img>, or CSS. These events should have an affectBusyIndicator setter to allow authors to decide whether a particular fetch can affect busy indicator of UAs. Of cause, global fetch events can also be used to implement a progress bar.

The suggested API follows MutationObserver's style, because this may be more efficient for large amount of events:

var fetchObserver = new FetchObserver(onFetch);

fetchObserver.connect(window, { // connect to current window object
  contexts: ['fetch', 'xmlhttprequest', 'form'], // filter by [fetch's contexts](https://fetch.spec.whatwg.org/#concept-request-context)
  types: ['start', 'progress', 'end', 'error'], // filter by types of fetch events
  methods: ['GET', 'POST'], //  filter by methods
  nested: true, // also observe nested browsing contexts, e.g. iframes, workers
});

function onFetch(fetchRecords) {
  for (var i = 0; i < fetchRecords.length; i++) {
    var fetchRecord = fetchRecords[i];
    if (fetchRecord.type === 'start' &&
       /^http:\/\/somedomain\.net\/myapp\//.test(fetchRecord.url)) { // filter by URL
      fetchRecords[i].affectBusyIndicator = true; // inform the UA that this fetch should affect the busy indicator
    } else if (fetchRecord.type === 'progress') {
      // compute and update the progress bar
    }
  }
}

[duanyao]

Yet another problem is: whether allow non-network IO operations (e.g. FileReader, IndexedDB, offline cache, etc.) affect busy indicator. For offline web apps, this seems a reasonable request. Maybe FetchObserver should be IOObserver?

[jussi-kalliokoski]

@duanyao "business" is not limited to IO either. Can be just processing in a worker. I wouldn't tie it to any API like fetch or even some generic IO API, it will fit more use cases as a standalone API and also be less complex.

[duanyao]

@jussi-kalliokoski
I got your point, but explicitly setting busy indicator has a few problems in my opinion:

• browsers currently don't use busy indicator for busy computation, so if a web app does, it seems counter intuitive.
• managing the state of busy indicator explicitly can be error-prone. If someone forgets to turn it off in a single code path, a web app may look busy for a surprisingly long time in some situation.
• currently it is hard or impossible to monitor IO operations that are not initiated directly by script, so if we want to implement a faithful busy indicator, this problem should be resolved first.

[Manishearth]

managing the state of busy indicator explicitly can be error-prone. If someone forgets to turn it off in a single code path, a web app may look busy for a surprisingly long time in some situation.

That's the webapp's problem. It's only going to affect that tab so we're fine.

[duanyao]

That's the webapp's problem. It's only going to affect that tab so we're fine.

Sure, but this makes the API hard to use despite it looks very simple.

[Manishearth]

I don't think this makes it harder to use. Not much harder anyway. There's much worse out there.

[duanyao]

I saw many codes using XHR just didn't handle error situations. So if those codes will use busy indicator in futrue, and a XHR fails, the busy indicator would on until next successful XHR.

[Manishearth]

If they're not handling error situations there probably already are going to be inconsistencies in the app (or the app is one where they don't particularly care). Why would this one be special?

[duanyao]

No, not necessarily be inconsistent. Those app may just stops updating the content when the network is unavailable, quite acceptable.

[Manishearth]

You're outlining a specific case where this would be okay; in general XHR is used for all sorts of things (eg displaying some new content on a click or mouseover), and in those cases the app would appear to not work anyway.

Also, note that webapps already implement their own progress indicators, and if XHR isn't handled properly, those can spin indefinitely too. I don't see a change in the status quo brought by access to the tab progress indicator wrt this problem.

[duanyao]

Also, note that webapps already implement their own progress indicators, and if XHR isn't handled properly, those can spin indefinitely too.

This is why custom progress indicator is hard to get right. This proposal is supposed to address this issue, right? If you give developer a tool looks very simple, but actually difficult to use correctly, the result is not ideal.

Managing the state of busy indicator explicitly is actually very hard.

• Overlapping XHRs are hard to handle. You can't simply turn the indicator on when starting a XHR, and turn it off when the XHR finished or failed; instead, you need a global counter of pending XHRs. This can be impractical if a app contains many modules maintained by individual parties.
• You have to manage the indicator(or the counter) at every places you call XHR, this is a big burden. If you are on a large code base, and someone makes a mistake when handling the indicator, it becomes a nightmare of debugging. Maybe you can write a nice wrapper for XHR/fetch that automatically manages the indicator, you still have to enforce the whole team to use it everywhere -- but how about third party modules?

So, I don't believe average developers can get explicit busy indicator management right easily, not to mention those who copy-paste code snippets from arbitrary sites.

[Manishearth]

If you give developer a tool looks very simple, but actually difficult to use correctly, the result is not ideal.

But the tool isn't the problem (the problem being "custom progress indicators get messed up if not done right") here, it's what the devs want to do -- and that's not changing.

Overlapping XHRs are hard to handle.

Again, already a problem, if the user does something to trigger multiple XHRs at once, a site not designed to handle it will stumble.

Most of the problems you list are problems with giving devs the XHR API; not with a busy indicator.

[duanyao]

Most of the problems you list are problems with giving devs the XHR API; not with a busy indicator.

It is not meaningful to blame XHR or developers here, because even the brand new Fetch API won't make busy indicator easier to handle.

The problem is the explicit state management. UAs already do good jobs at managing the state of busy indicator, why not reuse them? So I suggested FetchObserver and .affectBusyIndicator, and I believe this solution is hard to get wrong even if you copy-paste other's code blindly. FetchObserver also opens a door to many possibilities.

[smaug----]

In Gecko one can use a dummy iframe for this. iframe.contentDocument.open(); starts the busy indicator, and iframe.contentDocument.close() would stop it.
I assume other engines would have the same behavior if they followed the spec on document.open()/close() more closely.

But anyhow, I don't object adding API for busy indicator. Fetch spec of course wouldn't be the right place, but probably HTML.

[annevk]

It seems there is some interest from at least Mozilla in exposing this. I think the promise API proposed by @gaearon is the most promising. Maybe the constructor approach from @bloodyowl minus the progress bits (boolean seems fine given OS indicators to date). Though looking at https://w3c.github.io/wake-lock/ I wonder why @marcoscaceres ended up with just a property as API.

whatwg/fetch is probably not the best place to hash this out. If some people here would like to turn this into something I can create whatwg/busy so there's a better place to evolve this. Any takers?

(@duanyao you might be interested in #65. I don't think we want to tie a Busy Indicator API to fetching though. There's a number of other things such as WebSocket and WebRTC that are network-related and might cause a page to be busy. And I suppose we could even use it for non-network things.)

[realityking]

First of all, great news.

Promises are not a bad idea, but I can think of at least one use case only badly served with promises:

We're using angular-ui/ui-router in a number of Angular apps. Now if the library itself adds support, promises are easy as it works internally with (Angular-)Promises. However as a library user, all I have are the state change events which map relativly poorly to promises.

[domenic]

It's not a problem to adapt. addBusyPromise(new Promise(r => $rootScope.on('stateChangeStart', resolve))) is not much better or worse than var busyThing = new BusyThing(); busyThing.start(); $rootScope.on('stateChangeStart', () => busyThing.stop()).

[realityking]

Well it's tad bit more complicated, but you're right, it's not as bad as I though it would be:

$rootScope.$on('$stateChangeStart', function() {
    addBusyPromise(new Promise(resolve => {
        $rootScope.on('$viewContentLoaded', resolve);
        $rootScope.on('$stateChangeError', resolve);
    }));
});

[duanyao]

@annevk Now I also agree on the promise API proposed by @gaearon, and am glad to hear about #65. Thanks!

[igrigorik]

What happens if I have composite tasks with variable weighting? E.g. I have tasks to download and process and image, and I want to assign a higher weight to download stage because I know that (in absolute time) that will take much longer? How do I script that with above promise API?

Note that Apple recently revamped their API for progress indicators: https://developer.apple.com/videos/wwdc/2015/?id=232 - lots of good examples to think through there.

On a slightly different note..

• Not all UA's have a progress bar indicator, some use indeterminate indicators only
• If we expose an API we should make both progress and indeterminate cases possible
• We need to think through how such indicators overlap with browser triggered events...
  • Is this a separate indicator from the one browser uses?
  • Can/should the developer be able to control the browser indicator? E.g. call stop(), or some such, during page load to hide the indicator? (Not as crazy as it sounds)
  • Do the current browser indicators even make sense? e.g. onload is an anti-pattern.

Related, but on-topic rant: https://www.igvita.com/2015/06/25/browser-progress-bar-is-an-anti-pattern/

p.s. https://code.google.com/p/chromium/issues/detail?id=464377

[domenic]

This is a boolean progress indicator (the loading spinner), not anything with weighting or percentages.

[sicking]

I think so far people have mainly talked about the ability to turn on the progress indicator when the browser normally have it turned off.

Do we also need a way for the page to turn off the progress indicator once it has loaded enough stuff that the page "is ready"? See the paragraphs after the image in @igrigorik post.

[annevk]

This is now whatwg/html#330. Thank you for the suggestion @stevesouders!