Comments (10)
I think it is a matter of efficiency and testing what matters. In my library, I can test all relevant tests pretty fast, and Python is not known for its blazing fast speed π .
β coloraide git:(main) β python3 -m pytest tests/test_hsluv/test_snapshot.py
========================================================================================== test session starts ==========================================================================================
platform darwin -- Python 3.11.3, pytest-7.1.3, pluggy-1.0.0
rootdir: /Users/facelessuser/Code/github/coloraide
plugins: xonsh-0.13.4
collected 1 item
tests/test_hsluv/test_snapshot.py . [100%]
=========================================================================================== 1 passed in 1.28s ===========================================================================================
My comment from the test:
The snapshot contains a number of unnecessary tests, at least for our purposes. It is sufficient to verify that
our path from sRGB to HSLuv and HPLuv is correct, both forward and backwards, as our conversion path is noted
below:
```
srgb -> srgb-linear -> xyz-d65 -> luv -> lchuv -> hsluv
srgb -> srgb-linear -> xyz-d65 -> luv -> lchuv -> hpluv
```
Verifying each step in the chain makes no sense. If we are able to get through this entire chain and the sRGB
to HSLuv checks out, both forward and backwards through the path, then we can consider our implementation
successful. Any failure in the chain would cause errors to propagate up that would cause the test to fail. It
would only be useful to test each point in the chain if we were trying to identify where in the chain breakage
occurred.
You can see what I actually test here: https://github.com/facelessuser/coloraide/blob/main/tests/test_hsluv/test_snapshot.py
I would advise testing only the forward and backward conversion from HSLuv <-> sRGB and HPLuv <-> sRGB. Don't test XYZ as it is tested in the chain already. I wanted to formally pass the HSLuv and HPLuv tests to claim it was compliant, so I did not test all their data points for Luv and LCh, because again, if I can pass the HSLuv and HPLuv tests, I'm already passing LChuv and Luv. You can do a small set of Luv/LCh tests for completeness if you like.
One thing I'm uncertain of is whether part of the problem is that the test suite generates individual tests or each case generating excessive overhead? If so, I would consider maybe having a snapshot test, one test that validates the snapshot, so no additional overhead per case in the JSON. I imagine, even if I were to test every data point in the JSON in my own test suite, I could only see it increasing the snapshot test time by a little over double. So, I suspect some excessive overhead is generated by the test suite per test case in the JSON. But maybe I'm wrong.
from color.js.
Itβs great that we have so many tests for these new color spaces (16K tests!) but that means that now 99% of our testsuite is testing these two color spaces, and
npm run test
now takes several minutes (!).
That was the reason I initially had the tests in the hsluv
directory but somebody suggested moving them to the top level test
directory. π
I propose we pick a few characteristic tests to run with a regular testsuite run, and the rest can be run explicitly (via e.g.
htest test/hpluv/all.js
), when we really need to thoroughly test correctness for these color spaces.
I'm fine with that.
(also if there's anything on the hTest side that could improve that experience, I'm all ears!)
The majority of the time is spent updating the console with the countdown of the number of tests remaining.
More specifically converting result
to a string in this code causes the majority of the slowdown:
And I think this line in the toString()
method could be problematic because ret.children
might have 4096 tests (x4 if you're running the whole test suite)
from color.js.
I would advise testing only the forward and backward conversion from HSLuv <-> sRGB and HPLuv <-> sRGB. Don't test XYZ as it is tested in the chain already. I wanted to formally pass the HSLuv and HPLuv tests to claim it was compliant, so I did not test all their data points for Luv and LCh, because again, if I can pass the HSLuv and HPLuv tests, I'm already passing LChuv and Luv. You can do a small set of Luv/LCh tests for completeness if you like.
The tests only check sRGB to/from HSLuv and HPLuv.
from color.js.
Note that the test times reported by htest
are misleading and don't account for any overhead added by htest
.
from color.js.
Yeah, that overhead was what I was afraid was happening.
from color.js.
I hacked in some throttling to the node runner and the results look pretty good:
I did make some modifications to the hsluv tests to not parse color strings or create Color
objects which speed up the test times reported by htest
by around 3x.
Changes I made to src/env/node.js
:
done: (result, options) => {
let tree;
// Get the tree every 16ms or when there are no pending tests
if (result.lastUpdate === undefined || performance.now() - result.lastUpdate > 16 || result.stats.pending === 0) {
result.lastUpdate = performance.now();
let messages = result.toString({ format: options.format ?? "rich" });
tree = getTree(messages).toString();
tree = format(tree);
logUpdate(tree);
}
...
I also had to comment out this line in the same file because I was getting a "console.log is not a function" error.
//import console from "./console.js";
Changing from 16ms to 100ms made things slightly faster but less smooth.
from color.js.
Keep in mind that my PC is around 10 years old so those are probably close to worst case results
from color.js.
Good catch, throttling on the hTest side would probably be a good thing regardless. But even if perf was not an issue, I'm not sure having 8K tests testing a single color space is a good thing.
Note that the test times reported by htest are misleading and don't account for any overhead added by htest.
That is intentional, since you typically care more about how long your code takes to run rather than the test runner. They just don 't help in this case π
from color.js.
I'm not sure having 8K tests testing a single color space is a good thing.
I generally agree. I think it is a bit excessive. I assume it was done for the same reason I did, you are implementing a space, it has a test suite, so you verify against the test suite. If overhead is not a problem, it is completed fairly quickly.
But, I think at this point, we've verified the spaces, we can probably pick some data points at random as a sanity check, and call that sufficient maybe.
from color.js.
The conversion tests already contain a few tests for HSLuv and HPLuv so I'll just move the comprehensive test suite back to the hsluv directory.
from color.js.
Related Issues (20)
- Auto-generated API docs seem not to be using TypeScript files HOT 6
- How to have a single source of truth for types and docs? HOT 11
- 'https://colorjs.io/docs/spaces' doesn't always load the list of color spaces HOT 2
- Add interpolation support for OkLrCH HOT 7
- Color scales: gradient-like interpolation across multiple color stops HOT 4
- Scientific notation causes error in OKLCH? HOT 2
- new Color("--acescg").space.coords[i].range does not match coordinates of actual colors HOT 30
- OKHsl is on docs but actually not in the published version HOT 2
- Make it possible to serialize to the same format as parsed
- OkHSL/OkHSV does not use NaN for hue-less colors HOT 5
- Color.js is very error-happy
- Color space should be optional in `setAll()`
- Alpha is only clamped when parsing a color via the constructor, but not in `parse()`
- `color()` with more or fewer arguments should be invalid
- new Color("darkred").display() returns "rgb(0.5451 0 0)" HOT 5
- Newly broken tests HOT 7
- Customize `serialize()` output types without having to specify a whole format HOT 1
- New achromatic logic in LCH is not sufficient for all achromatic colors HOT 1
- Color.js typing for Coords is number even though functions may return null
- Thank you!
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from color.js.