studentofjs / query-plus

fetch and process data in web worker, store in indexedDB.

Query+

 

install

yarn add query-plus

or

pnpm add query-plus

or

npm install query-plus

import

import { useFetch, usePreFetch, FetchWithIntent } from "query-plus"

useFetch

useFetch is the main React hook of Query+.

examples

const { fetchWorker, data, error, loading } = useFetch()
useEffect(() => {
    fetchWorker({ url: 'https://api.test.com' })
}, []);

ProTip: You can use multiple useFetch() per component. Although you may be better off with a new component.

fetchWorker

const handleFetch = (() => {
fetchWorker({ url, preferUseCache, options, maxAge, middleware })
});

middleware

middleware is a function that takes the json returned from the endpoint and processes it.

let middlware = (data) => ({ name: data.name, powers: data.attrs.filter(attr => attr.includes("power") })
fetchWorker({ url: "http://api.test.com/trevor", middleware })

props

• middleware - optional function
• maxAge - the maximum amount of time stored data is valid in ms. If the maxAge has been exceeded, the data will be updated without checking for changes.
• options - same as the init variable from the fetch API
• preferUseCache - If this boolean is set to true and non-expired data exists in the DB, then the API will not be called.

usePreFetch

This React hook is used to pre-fetch data from any endpoints that you want to use later in your site. usePrefetch to pre fetch data and save it for faster retrival later.

example

usePreFetch([{ url: 'https://api.test/api/people/2/', middleware: (data: any) => ({name: data?.name}) }, {url: 'https://api.test/api/properties/10'}])

props

• url - the endpoint you want to fetch.

• fetch - init options.

• middleware - a function that accepts the json data, performs some action on the data and then returns it.

• maxAge: the maximum time in ms that the data remains valid.

useIntent

This is a React hook used for prefetching data based on a user's percieved intent.

example

function App() {
  const ref = useRef<HTMLDivElement>(null)
  useIntent<HTMLDivElement>({prefetch:[{ url: 'https://swapi.dev/api/people/3/' }],targetRef: ref, expandTarget: 100, timeToExcecute: 2000 })
  return (
    <div ref={ref}>
      <div className="App">
        <Droid />
      </div>
    </div>
    
  )
}

props

• expandTarget - optional number, increase the target boundary beyond your element.
• timeToExecute - optional time in ms required for the user to have their mouse cursor in the boundaries of the target area before prefetch is initiated. Defaults to 1000.
• prefetch - required array of fetch objects. See usePreFetch above.
• targetRef - the refference of the element you wish to use as the target.

Why Query+

We created Query+ as a React hook library, to make fetching, processing and caching data as easy as possible.

our goals

• Simple to use

• Real world performance gains

• Works in modern browsers (sorry IE11 diehards, but it's offically dead)

• Tiny bundle size (4kB minfied and gzipped)

• Not all the things, just what makes sense.

Sometimes APIs are slow, sometimes you need to process the data before you consume it and occasionly you want to store the response for use later. These factors all have an impact on the performance and percieved performance of your site. Waiting for a slow api, stuck on a loading screen, this is a familiar and unpleasant problem. How about that filter and map transform you run on the data? This will add to the time your users are waiting behind a spinner. Do you have an api that usually returns the same JSON? Seems inefficient and a bad reason to keep your users waiting.

Query+ tackles these issues by storing your api responses and only updating them when the new response differs from the stored data. This comparison and if you choose the middlware function you pass in for processing the response, runs in a web worker freeing the main thread for clientside interactions or whatever you want to prioritise. It means that your data is available immediatly, yet you aren't going to miss any updates.

technical

Data fetching, data processing and data storage tasks run in a web worker. This is to leave the main thread available for more critical tasks, improving performance metrics and user experience.

• tasks run in web workers

• data stored in indexedDB

• ideally the data in the response should be below 100kB per request.

• storage limit - At least 1GB, but can be up to 60% of remaining disk space.

• react >=16.8.0

Thanks to Jake Archibald for creating idb-keyval. Enabling us to easily access indexedDB with a really nice promise based api.

query-plus's Issues

Latest deployment to npm v.0.7.0

Issues with deployment not including dependencies.
Use version 0.6.0
Build and deploy later

tech spike Cache api review

• limited to 50MB in Safari
• clears after 2 weeks in Safari
• issues on iphone???

Are there any performance benefits to using this as a first layer of cache, before the space runs out?

proof read READ.ME and fix spelling mistakes

Improve FetchWithIntent component

Ideas for improvements:

Use coordinates instead of mouse enter /mouse leave. This will allow for a larger area and we could do it by data attr instead of having to use a component.

• Use coordinates plus/minus x amount as the countdown zone
• Ensure there is only one countdown running at a time
• Convert setTimeout to promise

component to prefetch on intent

Create a component that:

• - takes the same props as usePreFetch + children + a time to execute
• - on mouseenter or focus starts timer
• - on mouseleave or blur stops timer
• - initiates prefetch when the timer exceeds time to execute
• - manages timers carefully

allow requests to be marked as don't call unless expired

We are currently calling the api, regardless, to get the data to diff it against what is saved. If the endpoint always returns the same data, or updates on a semi regular basis (i.e. monthly) then it would be desirable to set the maxAge and not call the api if it hasn't expired. However, we will need a way to differentiate between this and endpoints where it would be desireable to check the api for new data on every call.
There appears to be two obvious solutions:

1. add a variable to the request
2. create a seperate function for fetching data, that only calls the api if none exists

I feel the first option is easier to use, but more prone to mistakes.

usePreFetch documentation

Update READ.ME with instructions and examples for the usePreFetch hook.

useFetch tests

Add tests for useFetch hook. react-hook-testing library or maybe just react-testing-library. Unsure on mocking indexedDB and a web worker, possibly a step too far