Comments (3)
Running into this properties limit is something that is happening to us in production. If this a change y'all would be interested interested I'd be happy to own the implementation as I'll likely be making this patch regardless.
Mainly would like to know if I should bother going through the effort of updating storage adapters urql ships and making a merge-able PR.
from urql.
I think the problems here we'll run into are that the format isn't detached from Graphcache of course:
urql/exchanges/graphcache/src/store/data.ts
Lines 610 to 657 in 8ff4e3e
For context (i.e. readers who'll be following this thread passively), this means that we'll have to make all storages incompatible and this will break any offline caches that someone may have already built-up. In short, this is definitely a big change, and a breaking change, if we consider storages to be a public interface; personally, I do, but this also depends on how many custom storages we expect people to have written, re.
urql/exchanges/graphcache/src/types.ts
Lines 933 to 935 in 8ff4e3e
Currently, the problem of storage persistence runs rather deep and I'd consider the current implementation optimised only for the format it has.
In short, it:
- buffers keys that have changed
- splits the keys back to their entity + field representation
- reads the value from the above entry (which is necessary to get the "ground truth" i.e. read all layers)
- marshalls the value and writes it to a buffer
- sends the buffer off to
storage.writeData
The reason this is done this way (besides reading from the layers in the "ground truth" order), is to not make writing more expensive than it is. We don't want the write performance to be impacted, which is why a buffer is created rather than "streaming" the changed values into a buffer/storage.
This also has the benefit of the process being deferrable.
I'd hate to lose this and while it's tempting to make this an iterable, it's probably not going to be performant, and we don't have benchmarks for this yet. However, any iterable-like could avoid key splitting and hence avoid an intermediary format.
The alternative of maps is interesting, but indeed causes serialisation issues. Hence, it doesn't actually solve the problem but pushes it onwards to storages. This increases complexity for storages.
So, the format per se isn't necessarily the problem:
- The problem is that
hydrateData
is an atomic operation that also doesn't operate on chunked iterables, which also would have benefits forpersistData
- We're currently only at risk of hitting this limit if the persist "chunk" is too big, or because
hydrateData
isn't chunked
So, this does open up more options and more aggressive changes than just switching to Map
s. Basically, if we can make the entire thing chunked by default and evaluate how to reduce some of the key and marshalling complexity then we may even get an even better interface here
from urql.
Firstly, thank you for the detailed response and all the work on this library ❤️
I did get Graphcahce working with Map
s, Mookiies@fcb9d11. However, I agree that simply switching to Map
s wouldn't be a good API change for the library. It's a huge breaking change and there were plenty of footguns in Map serialization for the StorageAdapter
. I didn't do any performance analysis. While it does solve my immediate issue of Hermes's low object properties limit, that's really a Hermes issue not one with Urql.
Coming up with an interface that would "chunk" by default would be awesome, especially if it can provide Graphcahce with performance benefits without significantly increasing complexity on StorageAdapters
. Have these issues around hydrateData
and persistData
only operating on large chunks ever come up in a different context as-well?
from urql.
Related Issues (20)
- Possible memory leak when using multiple queries on the same page HOT 7
- requestPolicyExchange does not re-request in production HOT 3
- cacheExchange - cache.readQuery (updateQuery) can't find queries with aliased variables HOT 1
- @urql/exchange-persisted Module name, 'crypto' does not resolve to a valid URL. HOT 7
- Non RSC authExchange failing HOT 2
- Problem: @urql/exchange-graphcache updateCacheWithResult reads null after writing link in cache updater. HOT 8
- Invalidating cache in Graphcache doesn't trigger query refetch in Next.js app router HOT 2
- RFC: react native file upload support HOT 8
- Introspected schema from urql-introspection doesn't match type of exchange-graphcache HOT 1
- RFC: Graphcache strong typing HOT 3
- Graphcache: mutations with different files in parallel, results lost HOT 3
- Invalidate with a key does not produce expected result HOT 2
- RFC: Normalize embedded documents based on parent HOT 5
- Syntax Error: Unexpected token in SelectionSet HOT 1
- RFC: @urql/vue - Allow getter syntax in `useQuery` HOT 4
- RFC: Add secondary keys HOT 2
- @urql/exchange-graphcache returns empty data & error although fetch response contains valid data. (when same entity is requested in parallel) HOT 1
- Multiple requests when using react suspense HOT 13
- ExecuteExchange doesn't interact well with "server-only" code HOT 1
- (vue) useQuery pause does not update when passing a ref HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from urql.