Comments (9)
I mean by (1) doing the the work in getCounts
in chunks of the data
at a time.
I have fiddled with (1) and it looks promising, I'm going to explore (2) and discuss further.
here's a proof of concept:
async function getCounts(
data: Uint8ClampedArray,
ignore: string[],
pixelsPerChunk = 1000 // what's an ideal default ??
): Promise<[]> {
function getCountMap(index: number, countMap = {}, resolve: Function) {
let upperBoundary = Math.min(index + 4 * pixelsPerChunk, data.length)
for (
let i = index;
i < upperBoundary;
i += 4 /* 4 gives us r, g, b, and a*/
) {
let alpha: number = data[i + 3]
// skip FULLY transparent pixels
if (alpha === 0) continue
let rgbComponents: number[] = Array.from(data.subarray(i, i + 3))
// skip undefined data
if (rgbComponents.indexOf(undefined) !== -1) continue
let color: string =
alpha && alpha !== 255
? `rgba(${[...rgbComponents, alpha].join(',')})`
: `rgb(${rgbComponents.join(',')})`
// skip colors in the ignore list
if (ignore.indexOf(color) !== -1) continue
if (countMap[color]) countMap[color].count++
else countMap[color] = { color, count: 1 }
}
if (upperBoundary == data.length) resolve(countMap)
else {
// queue the proecessing of the next chunk as a macrotsk
// for the next tick of the event loop
setTimeout( // what about requestAnimationFrame instead ??
() => getCountMap(index + 4 * pixelsPerChunk, countMap, resolve),
0
)
}
}
const countMap = await new Promise(resolve => getCountMap(0, {}, resolve))
const counts = Object.values(countMap) as []
return counts.sort((a: any, b: any) => b.count - a.count)
}
from rgbaster.js.
Hi,
On the topic of chunking out the work.
Rather than guessing how many to chunk at once, it's also possible to let elapsed time be the guide for each chunk and to let it process as many as it can in a given time interval. This somewhat automatically guarantees browser responsiveness regardless of how CPU intensive the iteration is. So, rather than passing in a chunk size, you can pass in a millisecond value. This of course does open the question of how many milliseconds to pick.
On a typical laptop with a refresh rate of 60 Hz, each frame is on the screen for about 16-17 ms. Lets assume we take a timer of 10 ms. This delay means a roughly 66% chance of blocking the ui for a single screen refresh. Such a delay would not be noticeable under most conditions. If the page has lots of animation, a 10ms delay might be noticeable. A gamble but not a huge one IMHO.
Here's an example:
// last two args are optional
function processLargeArrayAsync(array, fn, maxTimePerChunk, context) {
context = context || window;
maxTimePerChunk = maxTimePerChunk || 10;
var index = 0;
function now() {
return new Date().getTime();
}
function doChunk() {
var startTime = now();
while (index < array.length && (now() - startTime) <= maxTimePerChunk) {
// callback called with args (value, index, array)
fn.call(context, array[index], index, array);
++index;
}
if (index < array.length) {
// set Timeout for async iteration
setTimeout(doChunk, 0);
}
}
doChunk();
}
processLargeArrayAsync(veryLargeArray, myCallback);
I think we need to profile this code a little more to start with. I'm thinking that a 4k image would be the worst thing someone might reasonably throw at this. That's about 35mio items in the array to process.
from rgbaster.js.
Looks like we need to include the build files here: https://github.com/briangonzalez/rgbaster.js/blob/master/package.json#L27
Also, would you mind including the image you're processing?
Cheers.
from rgbaster.js.
Hi @briangonzalez,
Not sure I follow:
Looks like we need to include the build files here:
Whatever we add to files
in the package.json
will be in the tarball that is created when the downstream project runs npm publish
. Running npm pack
on RGBaster to get the tarball for inspection I get the following:
> npm pack
npm notice
npm notice 📦 [email protected]
npm notice === Tarball Contents ===
npm notice 1.1kB package.json
npm notice 2.2kB README.md
npm notice 2.8kB rgbaster.min.js
npm notice === Tarball Details ===
npm notice name: rgbaster
npm notice version: 2.0.0
npm notice filename: rgbaster-2.0.0.tgz
npm notice package size: 3.0 kB
npm notice unpacked size: 6.1 kB
npm notice shasum: 7c21609febb4180ff52b40d9208269a90dc5425d
npm notice integrity: sha512-rgQaLpeKUTfOQ[...]Mb9pXzy2ywakA==
npm notice total files: 3
npm notice
rgbaster-2.0.0.tgz
This looks to me like what I'd expect. I haven't set up an Angular project for further testing to see what would happen on a downstream project. Do you expect a different behaviour (ie. something other than inclusion of rgbaster.min.js
) ?
Maybe it's a good time to start thinking of ugrading briangonzalez/jquery.adaptive-backgrounds.js to the next version.
from rgbaster.js.
Looks like we need to include the build files here: https://github.com/briangonzalez/rgbaster.js/blob/master/package.json#L27
Also, would you mind including the image you're processing?
Cheers.
i will try to add files section in my package.json configuration. thanks for your help.
And the image i try to process is not so big.
Happy new year to you!
from rgbaster.js.
@AlfredJKwack On second glance, you're right. rgbaster.min.js
is the only build file.
from rgbaster.js.
I think this issue is the most pressing one before the next release. This library does indeed block the UI while doing its thing.
I suggest we look at 2 options:
- time-slicing with async recursion or otherwise
- web workers
or both of them.
@briangonzalez ?
from rgbaster.js.
I like #2 as it seems like the right tool for the job.
rgbaster is already async because we use onload
, so it won't even be a breaking change.
from rgbaster.js.
the image maipulation is a CPU intensive task, and it already takes a lot of time, even when choking the main thread, I think, as @briangonzalez mentioned, the web workers option would be better, giving us the freedom to process every thing in one loop without worries.
there are two options for this:
- export the
getImageData
andgetCounts
functions to the user, and document how they would string them together with anew Worker('./someScript.js')
that would contain some logic to executegetCounts
- workerize the
getCounts
function at build time using something similar to greenlet, I tried to include it butmicrobundle
doesn't seem bundle it correctly, also the tests fail with an error telling me that the modules default export is not a function !
ps: it seems thatpacel
does have the issue in (2)
Any thoughts ?
from rgbaster.js.
Related Issues (20)
- Performance [suggestion] HOT 8
- Module support HOT 1
- Many deprecated dangerous modules still in use HOT 2
- import node module in js file HOT 4
- Transparency etc HOT 5
- Path to v2 HOT 2
- v2: Missing test for Ignore color HOT 2
- V2: Scaling at 0 or >1? HOT 1
- v2: Missing test for promise.reject(error) HOT 2
- Thanks for the beautiful work over the last few months HOT 1
- gitignoreing package-lock.json HOT 2
- formatting code with prettier. HOT 1
- CORS test inverted in v2 HOT 1
- Could you Elaborate in Readme? HOT 4
- node js version. HOT 1
- Count value HOT 1
- TypeError: rgbaster_1.default is not a function
- Ignoring all rgb above rgb(180,180,180) HOT 2
- Can you publish 2.1.2 with the add `types` in package.json?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rgbaster.js.