Comments (13)
Looks like today is an HTTP Archive day. Will get onto it.
from legacy.httparchive.org.
Now I understand promises more 😀
I'll raise your simplification:
function fetchWithTimeout(url) {
var controller = new AbortController();
setTimeout(() => {controller.abort()}, 5000);
return fetch(url, {signal: controller.signal});
}
from legacy.httparchive.org.
Cool. Working on the last one now. sass.
from legacy.httparchive.org.
Synced the HA server with the changes in #193 so this should take effect in the October crawl starting tomorrow. Thank you again for hopping on this @Tiggerito 🙏
from legacy.httparchive.org.
I'm happy to reduce it for the robotstxt one. If it takes 5 seconds to return a simple text file, I'd classify that as an error on its own.
from legacy.httparchive.org.
Thanks @Tiggerito. Are you able to take all 3 files? Should be the same pattern in each.
from legacy.httparchive.org.
Thanks @Tiggerito. Are you able to take all 3 files? Should be the same pattern in each.
I'd have to research how to do it. This looks like a neat solution that could be put in a shared place. I think I saw that we can include js files?
https://www.lowmess.com/blog/fetch-with-timeout/
I could test it with my metric first?
from legacy.httparchive.org.
Here's a prototype of the JS I had in mind:
fetch = new Promise(resolve => setTimeout(resolve, 30000, 'fetch'));
timeout = new Promise(resolve => setTimeout(resolve, 5000, 'timeout'));
Promise.race([fetch, timeout]).then(value => console.log(value));
Shouldn't need to include external JS to do it.
from legacy.httparchive.org.
I tested using this (from the article I referenced) in WebPageTest and it worked well:
const fetchWithTimeout = (uri, options = {}, time = 5000) => {
const controller = new AbortController()
const config = { ...options, signal: controller.signal }
setTimeout(() => {
controller.abort()
}, time)
return fetch(uri, config)
.then((response) => {
if (!response.ok) {
throw new Error(`${response.status}: ${response.statusText}`)
}
return response
})
.catch((error) => {
if (error.name === 'AbortError') {
throw new Error('Response timed out')
}
throw new Error(error.message)
})
}
If I set a small timeout it returns:
{"message":"Response timed out","error":{}}
Which we could easily alter. Do we have a standard thing to return when custom metrics fail?
One advantage of this pattern is that it cancels the request on timeout, so no risk of having forgotten requests continuing to be processed.
It's also easy to plug in. Add the code and change the fetch(url) to a fetchWithTimeout(url), and it works.
from legacy.httparchive.org.
Well not to play favorites (I'm totally playing favorites 😁) but the Promise approach can also be implemented as a fetchWithTimeout
function and is much simpler:
function fetchWithTimeout(url) {
var network = fetch(url);
var timeout = new Promise(resolve => setTimeout(resolve, 5000, 'timeout'));
return Promise.race([network, timeout]).then(r => {
if (r == 'timeout') return Promise.reject(r);
return r;
});
}
from legacy.httparchive.org.
Hey @Tiggerito sorry for the delay, your function LGTM. Are you able to apply that to each fetch
instance? Hoping to get this in today before the October crawl starts.
from legacy.httparchive.org.
Testing the code now.
third-parties.js contains a fetch but is auto generated code built by bin/library-detector.js using what looks like another repository. It looks like the fetch is used in relation to the serviceWorker. Not a trivial one to alter.
Only thing I can think of is to update the builder to include code that intercepts the fetch. Something like:
let originalFetch = fetch;
fetch = function(url, options) {
var controller = new AbortController();
setTimeout(() => {controller.abort()}, 5000);
options.signal = controller.signal;
return originalFetch(url, options);
}
from legacy.httparchive.org.
These should be the only custom metrics with fetch: https://github.com/search?q=fetch+repo%3AHTTPArchive%2Flegacy.httparchive.org+path%3Acustom_metrics&type=Code&ref=advsearch&l=&l=
The code that generates the third parties script uses fetch, but it's not part of the custom metric code itself.
from legacy.httparchive.org.
Related Issues (20)
- Legacy website explorer limited to July 2018 HOT 2
- Crawlid 558 missing from stats HOT 3
- Legacy Website Reports are Missing Historical Data HOT 3
- Update FAQs HOT 1
- Video summary needs updating HOT 3
- Data from 2018-12-15 contains duplicate tests for some sites HOT 4
- Data from 2019-01-01 contains unknown crawls HOT 13
- Calculation of reqTotal incorrect for many sites for 2019-03-01 data HOT 1
- Legacy website not reachable HOT 4
- stats download for November is empty HOT 5
- A11Y metrics bugs HOT 3
- wpt_bodies meta description and robots gathering is invalid as the selector used is case sensitive HOT 1
- Create documentation file listing the contents of each custom metrics file
- Better script element custom metrics
- Create new scripts to detect importScripts() and usage of SW methods inside pwa.js HOT 1
- New event-names and pwa metrics did not use JSON.stringify HOT 7
- Add nativeSource to a11y custom metric
- Improve avif detection
- Improve a11y metric for captioned tables HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from legacy.httparchive.org.