Comments (35)
There's no suggestion of there being bugs in assertion libraries. I think there's a miscommunication. When I say that bugs might hide in thing.must.have.ownKeys(['foo', 'bar']) it's not that I suspect the implementation is faulty.
Ah, my bad, i did misunderstand you there. I actually agreed about this. It is actually usually even worse and looks something like 'smth.smth.smth.must....'
To anyone who heard of the Demeter's law this thing smells bad. And that is partly a reason why i slightly favour the 'expect' syntax. It is somewhat functional and doesnt fall on its face on undefineds.
Although coming to your original question which one produces a more bug free result, i think that both assert and should will fail equally. It just the should will fail with 'undefined doesnt have property named "must"'. Which sucks, but hey it is javascript, i shouldnt really complain :)
from dendritic.
For custom error messages, there is the message
arg in the constructor (https://github.com/moll/js-must/blob/master/doc/API.md#Must). Only available through the function-syntax though:
var demand = require("must")
demand(undefined, "The undefined undefineds").be.undefined()
I don't think the Demeter thing applies, or should apply, to reading properties out of nested objects/types. If it did, denormalization would the "solution". There is reason behind calling out modifying internals of a parent object through its nested objects, but for reading, I wouldn't raise it.
from dendritic.
I don't really have a strong inclination here tbh. I often find myself going for the expect
as I find it has both, assert
and should
parts in it. but that's more of a habit than a real preference.
one thing about asserts that bugs me a bit is that it's a bit clunky and inconsistent. i feel like expect(smth).to.eql(smth)
is cleaner than assert.deepEqual(smth1, smth2)
, it's just flows with the BDD way of wording things out nicer than asserts in my opinion.
but, as i said i don't really care as long as it is consistent throughout a project.
make it configurable?
from dendritic.
I prefer fluent assertions, and I prefer must. One thing to consider is that by promoting and using must we're also helping promote one of our own. That's a nice thing to do :)
from dendritic.
It's certainly a nice thing to do, but the core of the decision has to be what's the best solution. There are plenty of my own projects we don't use because they're not the best thing for the job.
The extra syntactic magic in assertion libraries is just more places for bugs to hide. What exactly does .toExist
test again? Is it falsiness or does it test typeof === 'undefined'
? It's errors in memory or understanding like that that lead to a green test suite with a broken codebase. That's a really hard class of bug to recover from.
assert.equal(typeof 'foo', 'string');
is completely unambiguous. Test code should be the dumbest, most pedestrian code you ever write. It should be absolutely unimpeachable in its function and its intent. It's okay for it to be verbose, it's okay for it to be clunky. It's never okay for it to be obtuse.
from dendritic.
that's a great opinion. and i'm fully agreed with what it says, but i differ a bit in terms of interpretation.
- I wouldn't worry about stability of a 1000+ stars assertion library. I am vastly more concerned about my own code than the assertion library i use. I know that in 99.9999% of cases it will my fault.
- asserts while dead simple are a bit limited and tend to obscure the intent. like in your case it will fail saying
array is not equal string
, which is not what you're actually testing there. - Asserts can be chunky. We had
equal
andeql
since Java 1.4..deeplyEqual
makes me cringe a bit, like it was the first assertion library that person used in their life.
Let me summarize it the old fashioned way with pros and cons:
asserts
Pros
- Dead simple
- Available out of the box
- Used since 1999
Cons
- Limited in terms of granularity
- Tend to obscure the intent
- Can be hard to read as english like text
Shoulds
Pros
- Very compact syntax
- Expansive and granular, allows to specify intent clearly
- More readable than asserts, although can feel weird
Cons
- Pollutes the prototype
- Explodes when called on
undefined
Expect
Pros
- IMHO the most readable of the three (as english like code)
- Expansive and detailed assertions
- Follows the BDD spec syntax guidelines
- Doesn't pollute the prototype
- Doesn't explode on undefines
- Used by majority of devs in different languages
Cons
- Mostly hated by
assert
people and creates tension - Can be seen as a bit verbose
That's basically my opinion on those three. Although I'm not a huge fun of expect
and it took me some time to get over it. I find it to be practical and stable.
Overall I prefer to see:
expect(smth).to.be.a('string');
expect(smth).to.eql({bla: "blah"});
than
assert.equal(typeof smth, 'string');
assert.deeplyEqual(smth, {bla: "blah"});
In my opinion the first one is a bit more compact, more clearly describes the intent, and it follows the BDD conventions.
again, my opinion is not that strong, the differences are rather marginal to me. but, i see this sort of reasoning a lot. when rspec converted from should to expect, we spent months arguing about it back and fourth. in the end the expect syntax won by a small margin and for pretty much reasons I have just described.
i just want to stress out that a lot of people feel rather strongly about this sort of things. and i'm fully respectful to that. hence, i suggest that this would be configureable. i don't really see it worth locking people in one way or another
from dendritic.
It's certainly a nice thing to do, but the core of the decision has to be what's the best solution. There are plenty of my own projects we don't use because they're not the best thing for the job.
I did not imply we should be using personal projects just because we made them. If we are to use an assertion library, it would be great to use a well built and tested one by a colleague where which we can help promote and improve the product. Using the best tool for the job has always been clear, and I haven't suggested otherwise.
The extra syntactic magic in assertion libraries is just more places for bugs to hide. What exactly does .toExist test again? Is it falsiness or does it test typeof === 'undefined'? It's errors in memory or understanding like that that lead to a green test suite with a broken codebase. That's a really hard class of bug to recover from.
I haven't experienced an issue like this before?
I get the point you're making, but I don't think we should avoid assertion libs. Be wary of issues like this, sure. But I've yet to have an issue with Must along the lines of what you're describing.
from dendritic.
I'd like to point out that issue #38 may affect the assertion library or lack there of we go with.
from dendritic.
Possibly. must
supports assertions on promises out of the box.
from dendritic.
so in order to work on #37 we need to come to a conclusion here. i recon there are two main issues.
- which style of assertions do we prefer to use,
assert
,should
orexpect
? - which exact assertion library we use for that?
from dendritic.
@davidbanham clearly prefers assert
am i correct? but how much of the actual applications code is he going to write?
also @nwinch i understand wants to use must
over chai
i kind of don't care that much about it. but my opinion is that chai
is a pretty much an industry standard at this point and must
doesn't provide any significant value on top of chai
. i don't mind the switch, but i think we need to acknowledge that we are stepping off the mainstream with this and every next person we hire will have to go through this. also we need to think about our clients who we will eventually hand over the product. would they be thrilled about the change as well?
also i'm slightly inclined towards the expect
syntax. again because it is a standard BDD syntax and used outside of javascript context. which makes sense to me in conjunction with mocha bdd environment.
@s-taylor was kind of quiet about the whole thing. and @lyntco was left out of the discussion completely
so can we all voice out our preferences here in maybe like:
- I prefer - blah
- I don't mind - blah
- i don't like - blah
from dendritic.
Yeah, I think we have more votes For than Against using an assertion lib, so as not to block progress I think we should use it.
Regarding which one, I prefer the should
style of assertions. Which must
supports. I like the fluent style it promotes and it is more readable to me. Both should
and expect
styles are very similar though, I'd say should
wins out for me purely on a personal preference.
I agree that chaijs
is used more widely than must
. More github stars, which implies more projects with a wider reach, and most likely more contributors. That is something we should strongly consider when weighing up speed of adoption.
I think must
is technically a little stronger in the battle, as it has a few things - such as promise support - baked in. Also this. We've also used this on many projects so far with positive results so there is a level of familiarity that exists.
@MadRabbit We need to be careful about using "the community does X" argument style here, as it's relative to each developer. I agree that we should wisely consider what our perception of the community is doing, though for all of us, that will be different. That said, I do see chaijs
used quite a bit in my travels around the open source country.
from dendritic.
I prefer Must, but I only have tape as a point of reference, I've never used chai.
Must supports promises out of the box which I think is a pro if we're using promises everywhere.
I don't think the learning curve on must is high, yes it's slightly different to Chai, but it's not hard to pick up.
It also recognises the work of one of our devs (Andri who wrote it) and promotes more of our code which is nice.
All that being said, if everyone wants Chai it wouldn't bother me either.
from dendritic.
@nwinch absolutely. the point of this is not to define what a community does but to make sure that we are making a decision with a clear understanding of options.
So, I personally don't mind the should
syntax, and, I think that guy with a crouch camera knows what he does. so as long as we understand the consequences which include:
- possible dissonance with the future employees
- possible dissonance with a client
- possible lack of community support comparing to chai
I'm happy with the must
lib & should
syntax.
@davidbanham thoughts?
from dendritic.
I've been using assert
(node) when doing small things, and expect
(chai) when larger ish things.
I guess the nice thing about knowing the person who wrote must
is that we might not need as much community support because we can just ask him XP I enjoyed must
while I was on Ordermentum, but in the end my opinion is not so strong on any.
assert
and expect
probably come close mainly because I have used them the most, but learning to do a unfamiliar thing is always okay to me.
from dendritic.
We have some +1s for must
and some "happy/don't mind", so this seems to have nosed it's way to the head of the pack.
from dendritic.
ok, so, it seems that must + should syntax is the thing that will make most of us either happy or work as an acceptable solution. lets wrap it up at that
from dendritic.
To beat a dead horse, I never wanted to write a testing library. I just had to after seeing the following gems in other libs.
Quoting myself from a few years back:
I didn't create Must.js merely because of the property access design issue, although I do consider that to be such an awful decision that it makes me wary of other design choices of those projects. There was also a Chai.js's eql bug1 that was open almost half a year on something so elementary as comparing an empty object with an empty array that drove me up the wall. Should.js couldn't handle objects with valueOf2 for a long time. Bugs can occur (though such really shouldn't in testing libraries which should be exemplars of proper systematic testing), but poor quality of basics and slow responses aren't good.
As for a single assert
function vs higher level libs, I too like simplicity, but I think diffs are worth it. Can't get a "foo" must be "bar"
with assert(a == "bar")
. And once one goes down assert.(deep)equals
, one's already in abstraction land and then it's just a matter of TDD vs BDD API. With a limited stdlib like in JS, the other built-in assertions of a testing lib are convenient, too.
from dendritic.
@moll just out of curiosity why didn't you submit a PR with a fix instead of building a whole new library?
also, just for the future reference. what is your preferred assertion style at the moment, assert, should or expect?
from dendritic.
Redesigning APIs in backwards incompatible way isn't as easy as a quick PR. Chai is still with its property access years later. For small bugs, I'm all for PRs. Poor or not, design taste is a leadership problem not solvable by a PR.
In JavaScript, I find the RSpec/OO style (a.must.be.a.number()
) and BDD style for the test harness (describe
and it
) to be most convenient.
from dendritic.
If we're using a lib, I agree that must
is a better choice than chai
or should
.
Let's unpack the readability concern further.
describe('fromulgator', () => {
it('should flibber the gibbets', () => {
return gabble.fromulgate(gibbets)
.then(gibbets => {
assert(gibbets.flibbered, 'The gibbets should be flibbered');
assert.deepEqual(gibbets, gibbetFixture, 'The gibbets should be flibbered in the exact way we expected');
});
});
});
describe('fromulgator', () => {
it('should flibber the gibbets', () => {
return gabble.fromulgate(gibbets)
.then(gibbets => {
gibbets.flibbered.should.be.truthy();
gibbets.should.eql(gibbetFixture);
});
});
});
I argue that the addition of specific assertion error messages to assert
makes it much more readable than the should
syntax alone. It doesn't look like must
allows custom messages.
> must(true).be.false('OHAI THERE LOOK AT ME')
AssertionError: true must be false
Actually thinking about this further the fix for that is just an inline comment.
describe('fromulgator', () => {
it('should flibber the gibbets', () => {
return gabble.fromulgate(gibbets)
.then(gibbets => {
gibbets.flibbered.should.be.truthy(); // The gibbets should be flibbered
gibbets.should.eql(gibbetFixture); // The gibbets should be flibbered in the exact way we expected
});
});
});
Leaving the above in place for posterity, but I'm willing to declare that the assertion lib style and the raw-assert style are at least as readable as each other. If anyone wants to mount a case for expect/should being more readable than assert I'm open to it.
Definitely agree that the diffed output from must's .eql
is likely to be far more immediately useful than the output from .deepEqual
. It's a benefit, but I think the size of that benefit is limited. When I'm TDDing with that test I'm likely to have something like:
console.log('gibbets is', JSON.stringify(gibbets, null, 2));
Before my assertions. Which, yes, is gross, but its lifetime is pretty limited.
@davidbanham clearly prefers assert am i correct? but how much of the actual applications code is he going to write?
Most of it already, and into the future hopefully more than I currently am! That's not actually the point, though. The choice here isn't made solely upon personal preference. A vote on who prefers using what doesn't get us the answer we're looking for. It's about which direction is going to result in fewer bugs over the lifetime of this application, and the applications built with this application.
The discussion seems to have moved away from that core premise. I have asserted (seewhatIdidthere) that assert
is less prone to bugs due to:
- Simpler API
- No dependency tree
The counterargument appears to be summarisable into:
Must is less prone to bugs than assert since the improved developer experience of diffed output and richer types of assertions makes people more willing to write more tests.
Is there anything that I'm missing?
from dendritic.
i think you're missing the text in the it(....
block, that is where you supposed to put some docs about what this test is all about.
i recon that is a fundamental difference between TDD and BDD. the first does
function blah() {
assert.equal(one, another, "what it all means");
}
and the second one does this
it("does this thing because of that", () => {
one.must.eql(another);
});
the difference is that TDD docs the assertions, and BDD docs the behavior and the assertions are just to verify the behavior. as rule of thumb a good it
block in BDD has just one assertion line.
so, technically speaking you're trying to do TDD within BDD, which imho defies the point
from dendritic.
Completely agree that that text should communicate what your test is trying to achieve. If you're Doing It Right, that should be sufficient to also explain your assertions. Often, however, you find yourself needing to add some extra explanation for a specific assertion like "I'm checking X here because it implies the existence of Y and I can't check Y directly because of Z"
Anyway, I think the readability/explanatory text thing is a dead issue. The BDD style does an excellent job of explaining intent and code comments can fill in any little gaps that it leaves.
I want to focus the discussion on what I think is the core question. "Which direction results in fewer bugs?"
from dendritic.
i think, and that was my position from the very beginning, that there is no real difference between assertion styles when it comes to making sure that things are tested and the app is shipped with a lower probability of an explosion at a runtime.
the only difference between those three I think is the "aesthetics" of it. namely, if i need to work on this project tomorrow, how long it will take me to stare into the screen before I'll be confident to make changes?
that is why i prefer to be in a consistent BDD mindset. because it is fairly standard and can be grasped by another developer quickly.
for example tape + asserts is a fine option for small cases. but as the complexity grows you have to hack things around as both tape and asserts are limited. and as there is no standard way of hacking them, you end up with something different every time. and then you hate yourself everytime when you have to return to a project 6 months later, because all deps are moved on and half of your things explode, and you don't remember why you wrote them this way or another
from dendritic.
I disagree that there's no real difference.
I have asserted (seewhatIdidthere) that assert is less prone to bugs due to:
- Simpler API
- No dependency tree
Also:
for example tape + asserts is a fine option for small cases. but as the complexity grows you have to hack things around as both tape and asserts are limited. and as there is no standard way of hacking them, you end up with something different every time. and then you hate yourself everytime when you have to return to a project 6 months later, because all deps are moved on and half of your things explode, and you don't remember why you wrote them this way or another
We're not talking about tape here, just asserts. Can you provide an example of the kind of hacking you're referring to with asserts?
from dendritic.
Let's move the BDD/TDD discussion to Flowdock and continue the core question discussion here. I'll kick it off over there.
from dendritic.
i kind of disagree with your disagreement
- assertion libs are normally very well tested, and i'd be infinitely more concern about bugs in my own code
- from coding perspective
require("assert")
andrequire("chai")
are pretty much the same - considering the size of any ORM or babel, an assertion lib isn't going to make a dent in your deps footprint
as for hacking asserts, i used to quite often add assert.includes
, because the alternative
assert(list.indexOf(smth) !== -1);
almost physically hurts
from dendritic.
@nwinch i think TDD vs. BDD is the right context for this discussion. without it, this doesn't make much sense, then either one of this is just an ==
on steroids.
i think doing TDD style assertions within a BDD spec is a blasphemy, and hence the choice depends on this context
from dendritic.
Yep for sure, I can see the relevance and overlap, though I had interpreted Dave's message here:
I want to focus the discussion on what I think is the core question. "Which direction results in fewer bugs?"
As vanilla assert
vs assertion libs - is that correct @davidbanham?
Apologies if I've misinterpreted, I don't want to throw a spanner in the works, just thought we could split concerns if there are concerns worth splitting. Please carry on if I had that wrong.
from dendritic.
i kind of disagree with your disagreement
Brilliant! Let's hash it out.
assertion libs are normally very well tested, and i'd be infinitely more concern about bugs in my own code
There's no suggestion of there being bugs in assertion libraries. I think there's a miscommunication. When I say that bugs might hide in thing.must.have.ownKeys(['foo', 'bar'])
it's not that I suspect the implementation is faulty. My concern is that not every developer on the project will have an intimate understanding of what exactly ownKeys
does and that differing or incorrect assumptions about that may introduce bugs into our application.
considering the size of any ORM or babel, an assertion lib isn't going to make a dent in your deps footprint
Which is part of the reason we use neither!
assert(list.indexOf(smth) !== -1);
assert(!_.includes(list, smth));
Lodash triggers the same objection in me as assertion libraries, btw. Really rich API that lends itself to misunderstandings about implementation details. I let that one slide due to the amount of additional utility is brings being worth the potential for confusion. If we can prove the same with assertion libs I'm sold.
from dendritic.
You've interpreted my statement correctly, Nath. Nikolay disagrees, though, and I think his disagreement is reasonably founded. Let's keep the discussion all in one place.
from dendritic.
Forget it. Let's roll with Must. I think that lodash point has tipped me over the edge. We're trading one potentially confusing API for another. Assert can also be confusing in different ways. If people are this keen on BDD style assertions let's just go for it. I think vanilla assert still has the edge, but this discussion has convinced me that the gap is small enough that it's not a big deal.
from dendritic.
from dendritic.
from dendritic.
This one isn't LoD for me. The concern is that I'm a very stupid man with a short attention span for documentation. When it comes to test code, I'd rather have 5 lines of very pedestrian code that have a very obvious purpose than 5 characters of clever API call.
If I come across an assertion library (utility library, etc) I'm very unlikely to ever go and read all the docs for its API. Chances are I'm going to assume what it does based off the name, maybe have a quick skim of the docs and get bored.
I'm happy to use those kinds of libraries and helpers in my application code, because if I do something boneheaded I know my tests will pick it up, along with all the other stupid shit I did.
I'm reluctant to use Clever Things in my test code. If I've done something dumb in my tests... that bug is at least going to a human tester and probably to production.
from dendritic.
Related Issues (20)
- "utils" vs "lib" HOT 3
- Route paths using filenames as relative base HOT 7
- Build analytics HOT 5
- Revision id is not checked when updating HOT 1
- "definitions" schema returned via /schema HOT 4
- /schemas folder location HOT 3
- Add the soft-delete option for models HOT 3
- Last build time + commit HOT 3
- Validation errors do not show fieldname HOT 2
- Validation errors difficult to match to form inputs HOT 6
- Docker automation package
- Move the redbeard specific code from boilerplate into dependencies
- v5 Validations
- thinky.dbReady is borked in docker
- Tests are flaky due to random project/resource names
- app-module-path must go
- Update deps
- createdAt/updatedAt as default schema/model fields HOT 7
- Base Controller .update uses this.find which can be overwritten. HOT 4
- Interview partners for research about communication in GitHub projects wanted
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dendritic.