Snippet tests about 30-seconds-of-code

[skatcat31]

Actually that might be a really cool system to pull in new contributors! What about a learning collection of .md with a bunch of tests that people can code against for fun to satisfy the tests and learn how testing libraries work? Like 30 Seconds to Tests or something. Then poeple could come here to learn helpful snippets, and how tests work.

Now that is a large and hard project, but it also gives us an excuse to test older snippets and not new ones just yet 😉

[kingdavidmartins]

Hey All 👋 😄

So earlier last week I had an Idea. Thinking

"wouldn't it be nice to able to create some lib/package that could make adding/integrating a TDD to any project super simple & hastle free. Making life easier for both contributors/collaborators"

And after spending sometime thinking about it today. I decided to spend a little time brain storming on what could be done to keep the barrier to entry low all while being able to add/integrate a TDD evironment

I ended up comming up with an idea of a lib/package that works based on specified flags. I believe by adding flags the time taken to Integrate/Add TDD with existing/new projects can be reduced down by multiple factors and/or a factor of 10

I'll ellaborate through the following example

compact

Removes falsey values from an array.

Use Array.filter() to filter out falsey values (false, null, 0, "", undefined, and NaN).

/* PF: */ const compact = arr => arr.filter(Boolean);
/* PT: */ compact([0, 1, false, 2, '', 3, 'a', 'e'*23, NaN, 's', 34]) // [ 1, 2, 3, 'a', 's', 34 ]

/* PF */ stands for pull function & is the function that would be pulled in said environment. Which in this case would be

const compact = arr => arr.filter(Boolean);

/* PT */ stands for pull test & is the test that would be pulled in said environment and run in conjuction with the pulled function. Which in this case would be

compact([0, 1, false, 2, '', 3, 'a', 'e'*23, NaN, 's', 34]) // [ 1, 2, 3, 'a', 's', 34 ]

Users can apply specific tests by doing the following

/* PT:deepEqual */ compact([0, 1, false, 2, '', 3, 'a', 'e'*23, NaN, 's', 34]) // [ 1, 2, 3, 'a', 's', 34 ]

/* PT */ Take's the function(args...) and test it against what has been commented

What Our TDD Is Testing & Expecting This To Pass

deepEqual(compact([0, 1, false, 2, '', 3, 'a', 'e'*23, NaN, 's', 34]), [ 1, 2, 3, 'a', 's', 34 ]) 

What would then happen would be that the script/lib would look for said flags and BAM ⚡ just like that. Our said TDD environment has been generated.

All that would be left would be running.

npm test

& We'll be able to see all the functions/methods that passed and haven't.

As we can see this can significantly reduce the time taken integrating TDD into a existing project especially when you don't have to think about porting methods into your testing environment & adding methods to your project all while adding specific test to your test/**.test.js.

Thought's @skatcat31 @atomiks @Pl4gue @fejes713 @iamsoorena

[skatcat31]

However writing something to extract the code for testing purposes isn't a bad idea.

[Chalarangelo]

Ah, so you mean like a tag_database but for tests? I'd like that s lot.

[Chalarangelo]

As discussed yesterday, we are currently looking for an online tool where we can post our tests for individual snippets and share them (think jsPerf but for tests). If anyone knows of such a thing, kindly point us towards that direction.

Automated testing, splitting snippets etc. is not going to be added until we figure out a way to have it implemented without raising the barrier of entry for new contributors.

[skatcat31]

@kingdavidmartins what I mean is I'm not a fan of a file both having what would define the snippet, and what would define the tests to be extracted out into a .js file since the snippet file would become very noisy instead of a simple .md file, and we need to change another script heavily to support the feature.

This also bring in the possibility of multiple updates to the same .md changing the footprint of the lines and causing merge conflicts, that if not handled correctly would break the tests. Remember the larger and more complex a file gets, the harder it is to keep safe.

While it was an idea to possibly simplify things and bring in tests, it misses a big mark in simplification: Don't make users learn something specific to contribute. That only leads to one thing: alienating contributors because they don't know the secret handshake*.

---

It may be better to make a new folder for tests, where each folder is named after the snippet file and has an index.js that would want to co load the snippet from a file called snippet.js that would just be the snippet code to the exports of the file, and then a test script that loads the directory above, loops over every folder, and run all of the tests. No folder means no tests, means they don't get run unless we want to test the snippet.

This also allows you to go to an individual folder and run the tests for that snippet directly after the build script and not have to run all of the tests. All it would require is the build script to be modified to also check if the folder exists, and then put the template for exporting the snippet into the folder into the aforementioned snippet file.

This brings several advantages:

• No specialized parsing beyond what is already done
• No conflicts breaking tests because a snippet became a line longer in someones commits, and not another person's
• Easier to maintain because a missing tag doesn't break extraction
• Defined structure means defined incorperation
• A user does not need to learn our projects individual markup language to contribute, contribution are still just a singular .md file for a cool snippet
• No heavy script rewrites, just a simple script extension
• Tests have a very defined formula that is already similar to prior arts and best practices
• Allows automated updates of test based on changes to snippets without rewriting the test file
• Tests are write once, apply always, and update through direct changes instead of rewritten every commit, changed indirectly through consequences, and prone to failure due to outside sources**
• Shorter build time since tests don't always need to be re-parsed and rewritten every build, just re-run

There a plenty of reasons to keep tests separate from snippets.

*: Something specific for entry into a singular area
**: Primarily conflicts and missing tags in an update

[Chalarangelo]

After some discussion, we will start implementing testing sooner or later. I have assigned this to @kingdavidmartins who originally started this discussion and expressed interest in implementing it, along with @fejes713 who wants to help out with the implementation! Now that the restructure is live, this is our top priority issue!

[skatcat31]

@kingdavidmartins can you post a layout/outline of your testing architecture so we know how to help contribute to it before the generator is done?

[kingdavidmartins]

Hey @skatcat31 👋 😄

can you post a layout/outline of your testing architecture so we know how to help contribute to it before the generator is done?

Yea No Problem

Version 0.0.1

layout/outline

1. Check if test/ dir exist, if it does then empty the dir so it can repopulate.
2. Create a test dir
3. Create a dir/folder for each snippet name with the following format

• snippetName.test.js -> [creates seperation of concerns]
• snippetName.js -> the snippet will be exported to the test file for testing [creates seperation of concerns]
• import/require said snippets
• set up with 5 different example test that are commented out with a commented out link to all the other methods made available by tape

Currently takes 100ms to generate on my machine

Note: I don't think the generator will ever be done & will most likely be updated retroactively. Be it may for optimizing purposes, restructuring, new libs, etc

[kingdavidmartins]

Hey @skatcat31 👋 😄

We should clear out the snippet files. This way we can develop tests separately from the snippets

You honestly bring up a good point. That way contributors/collaborators can really develop a more rigorous testing suite that can still test a specific snippet the same way it did before or better rather than starting all over again every time the script generates. Nice catch 👌

What I would also have to do is have the script check if a test for a snippet exist. If it does it will ignore, if it doesn't it will generate and if a snippet was removed it will have to also make sure the associated snippet is removed.

as for

set up with 5 different example test

I meant the following. This would be the starting point generated for the associated snippet in the test/ folder in their respective dir -> test/snippetName/ -> snippetName.test.js

const test = require('tape');
const escapeRegExp = require('./escapeRegExp.js');

test('Testing escapeRegExp', (t) => {
  //For more information on all the methods supported by tape
  //Please go to https://github.com/substack/tape
  t.true(typeof escapeRegExp === 'function', 'escapeRegExp is a Function');
  //t.deepEqual(escapeRegExp(args..), 'Expected');
  //t.equal(escapeRegExp(args..), 'Expected');
  //t.false(escapeRegExp(args..), 'Expected');
  //t.throws(escapeRegExp(args..), 'Expected');
  t.end();
});

Thanks for the feedback

[skatcat31]

Awesome. Now if I could get over tapes syntax XD
I use mocha plus unit.js and they way they arrange tests are a little different, but recently I've just been writing my own test suites

[mariastervic]

Is snippet testing really necessary? Many snippets will not change after some time and testing them before submitting can make sure they work properly. Also a lot of snippets cannot be easily tested I think. I have used code from this collection and I have not found anything that is broken yet. Curation and code reviews for PRs should be enough.

[Chalarangelo]

@kingdavidmartins browser snippets are a small fraction of our codebase, let's iron out all the minor issues by adding tests for the rest of our snippets, then move on to the browser ones that require some extra care.

[skatcat31]

NEVER put tests with code. It means it's very easy to accidentally screw up your tests, and also that you aren't testing the interface to loading your code as well, which means you lose setup tests too.

Learned that the very hard way. While there is no hard set rule to not do that, once you screw up once, and how often you get merge conflicts if someone else updates the same file it just isn't worth it.

[Chalarangelo]

(not contributing to the discussion, just a heads up) Testing will be implemented after we have dealt with snippet restructuring (#233) and it will be our top priority at that time.

I will read through the rest of the discussion and tell you my opinion later. 😉

[Chalarangelo]

I agree with @skatcat31 in that separation of concerns and keeping the barrier of entry low are very important. Tests should contain the examples we specify for a snippet and a lot more that we need to test them internally. No reason to show all of those test cases to people who just browse the site looking for a useful piece of code.

[skatcat31]

So if I'm reading this correctly we finally decided on a structure for testing?

[kingdavidmartins]

Hey @skatcat31 👋 😄

It may be better to make a new folder for tests, where each folder is named after the snippet file and has an index.js that would want to co load the snippet from a file called snippet.js that would just be the snippet code to the exports of the file, and then a test script that loads the directory above, loops over every folder, and run all of the tests. No folder means no tests, means they don't get run unless we want to test the snippet.

I agree, that would have been the plan. All though running npm test would still test all loaded/generated snippets. One can also go to the snippet file and run the individual snippet test

Don't make users learn something specific to contribute. That only leads to one thing: alienating contributors because they don't know the secret handshake.

Agreed. Makes sense. No matter how "simple" it can still lead to alienating potential contributors

I believe the current goal is to get a script going that can auto generate our TDD.
I believe it will be a tool that will be mostly used internally to test proposed/current snippets more rigorously as stated by @Chalarangelo

Tests should contain the examples we specify for a snippet and a lot more that we need to test them internally. No reason to show all of those test cases to people who just browse the site looking for a useful piece of code.

Note: Version 0.0.1 will start things off and the tdd script will then be optimized retroactively.

[skatcat31]

@kingdavidmartins I see a simple problem with that setup: By clearing the test folder every time we end up not being able to expand the tests. We should clear out the snippet files. This way we can develop tests separately from the snippets

set up with 5 different example test

I'm a little confused by this part. Can you clarify it a bit?

[skatcat31]

equals true( val == true ). If you didn't return a truthy value I assumed you screwed up writing the test since you should know when you'll need to negate your return

[kingdavidmartins]

@Chalarangelo Yea for Majority ~ Nodejs based snippets Giving us a TDD coverage of 89%~90% 171/193 in regards to our project. However haven't implemented testing for Browser Snippets yet which covers the other 10%~11% 22/193

[Chalarangelo]

@kingdavidmartins Browser testing will probably be a huge pain, as Travis doesn't seem to have a browser installed and testing will be done mainly in the cron job. The best we can do is test browser snippets separately or find a tool that somehow works on Travis (I do not have high hopes about that).

[Chalarangelo]

Please join us on gitter to discuss this further. I'll be posting updates to this issue as we go. 😉

[mycaule]

There are also dynamic documentation generators.
In Scala we have: https://github.com/tpolecat/tut
Scala exercises is also interesting: https://www.scala-exercises.org/std_lib/lists, their project may give you ideas for the website.

You can also make a Python Notebook with a JS kernel. It is a live editor, where you can paste code and see results, then producing a scientific style report. Google Colaboratory might be the multiuser sharing tool that you are looking for, though it only supports Python for the moment.

I don't know if there is similar in pure JavaScript, I am going to check that later today.

[kingdavidmartins]

Hey @skatcat31 👋 😄

Thanks for the feedback. Highly appreciated

However I'm kind of confused with the statement

NEVER put tests with code

I am not proposing actually doing the following

deepEqual(compact([0, 1, false, 2, '', 3, 'a', 'e'*23, NaN, 's', 34]), [ 1, 2, 3, 'a', 's', 34 ]) 

But actually proposing

writing something to extract the code for testing purposes.

I believe it's my fault for omitting the actually output. But the code would actually be extracted to the test/ dir where the following would be generated in the respective test/**.test.js file

const test = require('tape');
const compact = arr => arr.filter(Boolean);
test('Testing Compact', (t) => {
  t.plan(1);
  t.deepEqual(compact([0, 1, false, 2, '', 3, 'a', 'e'*23, NaN, 's', 34]), [ 1, 2, 3, 'a', 's', 34 ]);
  t.end();
});

When I said

deepEqual(compact([0, 1, false, 2, '', 3, 'a', 'e'*23, NaN, 's', 34]), [ 1, 2, 3, 'a', 's', 34 ]) 

I didn't actually mean I would be putting test with code. I just omitted the proper expected output since I was just tired.

Hope this helps clears everything up

[skatcat31]

Ah that clears it up a little. Yeah that works, just know that putting it in the same file to be extracted later will lead to more conflicts as code and tests get changed or change in length through different contributors. Although I doubt that would really happen, it is the nature of people branching from co descendants in large projects( probably won't matter, but it's something to keep a mind of )

[kingdavidmartins]

@skatcat31 Agreed, It now become's making sure the extractor becomes robust enough to handle said edge cases & making sure when generating **.test.js files, each snippet has it's own file. Giving each snippet/method their own **.test.js file should make creating any conflicts highly improbable. Which in this case would be compact.test.js.

Also similar to how we have the following scripts
npm run builder
npm run linter
npm run tagger
npm run webber

We and/or the lib/package can also have a similar script like Example npm run gen-tdd
Which someone could run every time they update their repo.

So the test that npm test runs stays up to date similarly to how the tag_database & etc stays up to date.

[skatcat31]

I'm still a little wary of putting tests in the same file, and either way it's us passing things through eval( always a danger if we aren't careful )

[kingdavidmartins]

@skatcat31 what do you mean by the following?

and either way it's us passing things through eval( always a danger if we aren't careful )

I don't intend on passing anything through eval() or using eval() for that matter.

Also by the following statement what do you mean exactly?

I'm still a little wary of putting tests in the same file

Do you mean you are wary of doing this?

const test = require('tape');
const compact = arr => arr.filter(Boolean);
test('Testing Compact', (t) => {
  t.plan(1);
  t.deepEqual(compact([0, 1, false, 2, '', 3, 'a', 'e'*23, NaN, 's', 34]), [ 1, 2, 3, 'a', 's', 34 ]);
  t.end();
});

And prefer this?

const test = require('tape');
const compact = require('../snippets/compact')
test('Testing Compact', (t) => {
  t.plan(1);
  t.deepEqual(compact([0, 1, false, 2, '', 3, 'a', 'e'*23, NaN, 's', 34]), [ 1, 2, 3, 'a', 's', 34 ]);
  t.end();
});

or?

So I can understand your reasoning & concerns

[mycaule]

There is this tool called RunKit that provides JavaScript notebooks: runkit.com

RunKit is a node playground in your browser

It is also promoted by npmjs.com to showcase the packages.
For example, https://www.npmjs.com/package/express: right side menu Try it out

Example of Notebook: https://runkit.com/mycaule/30-seconds-of-code

Just like CodePen, RunKit has a runnable HTML snippet functionality
See. https://runkit.com/home#embed

You just have to write this to have an embed runnable JS script.

<script src="https://embed.runkit.com" id="my-element"></script>

<div id="my-element">
    console.log("hello world");
</div>

It should be easy to improve the HTML generator for https://30secondsofcode.org/ with this.

[Chalarangelo]

In general, I am against putting tests in the snippet files, as this would be quite a mess we would have to deal with. Tests should be in external files and we should automate testing using CI.

@skatcat31 I agree we should extract code from the existing architecture and keep testing separate to make sure everyhing is nice and tidy and also keep barrier of entry low without encumbering new contributors with the whole testing situation.

@mycaule I object to adding more overhead to the html page as it is already getting heavy to load on a bad connection (I happen to be on mobile during my vacation and notice it already has some issues loading on a reasonably bad 3G (or worse) connection). The heavier the page the more traffic we will lose. This is pretty much the reason why the copying to clipboard was implemented the way it was, although it burdens the browser with injecting the buttons (otherwise we would have to build them all via Node, which would be easier, but still would make the page heavier to load initially).

[mycaule]

Yes maybe keep things simple and light as they are currently.

During the build in web-script.js, may be you can extract out comments from your markdown snippets.

// primes(10) -> [2,3,5,7]

to test that

primes(10) === [2, 3, 5, 7]

and to ensure code quality just before publishing.

A library containing all the functions and test files could also be generated from the standard you described in the markdown snippets.

If the library and test files are generated but not committed, you can even plug this to npm test in package.json to have it verified by Travis CI and then allowing easy pull requests for new markdown files addition.

I can make a PR for this if you want.

But overall, I agree you have pretty decent stuff already!

[mycaule]

@kingdavidmartins point of view is about adopting the way they designed the build of https://github.com/lodash/lodash: splitting everything is small files, write tests in comments and piping the stuff in the build system.

[Chalarangelo]

Extracting and testing is definitely out of scope for the web builder. We want the build process to be fast and easy to debug. However, there can be an extractor script written to do this kind of processing.

[kingdavidmartins]

@Chalarangelo I'm so confused. If the following was a response to the flag suggestion. I never said I wanted to do the following

In general, I am against putting tests in the snippet files, as this would be quite a mess we would have to deal with.

I have been proposing been always and still proposing the following

I agree we should extract code from the existing architecture and keep testing separate to make sure everyhing is nice and tidy and also keep barrier of entry low without encumbering new contributors with the whole testing situation.

Once again the key word here is flag. As the extractor will be looking for said files with said flags so that the proper testing environment can be generated.

Nothing will be generated in the snippets/ directory, it will generated in the test/ directory
Similar to how scripts like tagger read snippets/ dir and add/generate new line for every new file that was updated/added to the snippets/ directory with the following format fileName: added to the tag_database or similarly to how webber works. The difference here will be that the intended script will be dependent on flags

Everything will be seperated and self contained.

[kingdavidmartins]

@mycaule Although your idea Is quite interesting I have to agree with @Chalarangelo statement

Extracting and testing is definitely out of scope for the web builder. We want the build process to be fast and easy to debug. However, there can be an extractor script written to do this kind of processing.

@Chalarangelo that's what I intended seeing that many other projects can also benefit from said idea. Depending on the web builder isn't feasible and/or portable. Would have to be It's own extractor script

@Chalarangelo Yess!! Exactly 🎉

[kingdavidmartins]

@Chalarangelo Although the behavior/use case would be different from the tagger script. It can definitely be simplified to the following statement

Ah, so you mean like a tag_database but for tests? I'd like that s lot

[fejes713]

@skatcat31 You are right! Testing is our main priority right now

[kingdavidmartins]

@skatcat31 Hahaha 🤣
Yea, I didn't want to add too much tooling. I think mocha + unit is kinda overkill for this project.
By your own test suites do you mean your own test runner, assert lib, etc?

[skatcat31]

Pretty much. I created a simple class that calls a callback with a function, and each call expects a string, and a CB or value that coerces to true or false, logs with colors, and then pushes the result. It then waits for the callback to return, and then logs an array of all tests and their results. Uses Promises and Promise.all. IT was a fun project but not one I use in production

[kingdavidmartins]

Hmmm that's interesting. What assertion style did you use?

[kingdavidmartins]

Ahh gotcha. Did you add any other functionality that was different from the other testing suites currently available? @skatcat31

[atomiks]

Are we going to use karma to test the browser snippets?

[Chalarangelo]

@kingdavidmartins It's been a few days, any progress on TDD? Do you need any help? Should we start writing tests for snippets?

[skatcat31]

@mariastervic Snippet testing is extremely useful in identifying places your snippet should theoretically work. After all if you call a string method and it doesn't work it would be odd that it doesn't. As it stands the only reason these currently all work is we'd tested them manually. With a test suite we can test against theoretical edge cases that a snippet shouldn't have hit

[kingdavidmartins]

Hey @atomiks 👋 😄

So far I've only used tapejs for testing with regards to/with our snippets which work perfectly well with our nodejs based snippets.

However as far as browser goes I've been thinking of using nightwatchjs

// Not only is it very readable, when test runs it opens a browser 
// which automatically goes to `www.google.com` and does/checks the following
// as described with methods below.
module.exports = {
  'Demo test Google' : function (client) {
    client
      .url('http://www.google.com')
      .waitForElementVisible('body', 1000)
      .assert.title('Google')
      .assert.visible('input[type=text]')
      .setValue('input[type=text]', 'rembrandt van rijn')
      .waitForElementVisible('button[name=btnG]', 1000)
      .click('button[name=btnG]')
      .pause(1000)
      .assert.containsText('ol#rso li:first-child',
        'Rembrandt - Wikipedia')
      .end();
  }
};

@Chalarangelo Hey 😄

Yea everything fine, Just been really under the weather and needed sometime. Feeling much better now. NY got hit with an Artic Blast 😰 & Blizzard the last few days.

Yea, So to answer your question, Yea you can start writing tests for said snippets after running npm run tdd. Testing them all by running npm test.

Just have to update the tdd script to work the updated structure testing Active Snippets & Archived Snippets which I'll submit a PR for

[Chalarangelo]

@kingdavidmartins So, in theory, after merging #516, this issue should be considered resolved as the architecture is ready to go and we are just missing the relevant tests for snippets?

[kingdavidmartins]

Hey @mariastervic 👋 😄

Exactly what @skatcat31 said. Especially if said snippets get updated and takes multiple arguments, said updates can cause some sort of unexpected behavior when a particular variable is or can be omitted it which can become more likely to fail through the cracks.

However if you already have tests you've put in place for a particular snippet that's very vigorous & checks said edge cases then all you'll have to do is just checkout the PR & run your test file. Easy Peasy

[kingdavidmartins]

@Chalarangelo Yea, I agree

The best we can do is test browser snippets separately or find a tool that somehow works on Travis (I > do not have high hopes about that).

I believe more like the latter seems more reasonable & feasible. Although there's a few tools that work with Travis CI ~ karmajs included we can build out the functionality of our TDD environment retroactively to include browsers & whatever else that is needed

[kingdavidmartins]

@Chalarangelo Makes sense!

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for any follow-up tasks.