prismatik / dendritic Goto Github PK

I've noticed that you hardcoded env vars into the autogenerated package.json. You kind of encouraging devs to put more in there.

I recon this should be solved in a more standard way by using standard confirguration and .env files support

should be really under the test/ folder in my opinion

This was a bit of an elephant in the room for me on this project and I feel like I'm missing something.

So, why exactly we're building our own templating and CLI thing when there is yoman?

yoman was a go to tool for this sort of things for a few years now. here is for example a MEAN stack generator

I feel like we are reinventing a wheel here and at the same time blocking ourselves off all the awesome things that community already built

So, I'm new to the tool and don't really know how to use it. I've just installed redbeard, and run redbeard and then redbeard -h and then redbeard --help. it exploded all three times. would be nice to have some help for the users.

As per this discussion, we are moving to Mocha. Closes #37.

This is a must have. It just plummets my performance if errors are not handled properly both in dev and prod.

I saw you added the unhandledRejects, but I don't think that cuts it. You need to have logged the errors somewhere like sentry or airbrake in prod and it should properly capture the context and stack trace. Also in dev it should show a proper and meaningful stack-trace + don't crash and automatically reboot when changes to the code are made

Nath and I were discussing yesterday some of the challenges around using the redbeard API and sockets in conjunction. For example when you want to create a record, currently you must use the API endpoint.

What would normally happen if you were just using an API is you'd POST your document and then use the success response to add the document to state. But since we're using sockets the new document should come through automagically without needing to listen to the response. But then if it errors we still need to do something?

An alternative might be to channel all the things through sockets? Is there any reason we can't send create, delete, and update requests via web sockets as well? We could also have web sockets emit error events for when a create, delete or update request fails. That will make things cleaner than working with two interfaces. We could also then allow the redbeard controller [name] command to scaffold just the API endpoints, the websocket interface or both. Thoughts?

There are currently no tests to check whether found: true is included in the resolved object from a call to get in the controllers.

Adding such a test found it breaks when params.count !== true, even if responses are found, I assume its this line here: https://github.com/Prismatik/redbeard/blob/master/controller/controller.mustache#L93

The tag probably should be result

As discussed in #28, add a function to the migration bin script to show you the most recent migration that has run against the target database.

Would also be nice to have an option to display all run migrations.

Right now orderBy is not using our indexes.
We're using .orderBy('date') in all cases rather than .orderBy({index: 'date'}) when an index exists.
We should optimise our query based on if the field requested to use orderBy is indexed or not (which is now identified in our schema).

https://www.rethinkdb.com/api/javascript/order_by/

The way controllers and routes are split don't make much sense to me and feel quite non-standard to be honest.

It feels like your controllers are more like a services layer, but they also barge into actual action controllers field. and then your routes are very repetitive and offload a lot of things they should do into what you call controllers.

i most modern projects i see and have been working on the structure is that routes mostly play the role of controllers that handle the HTTP data and responses. routes talk to services or models directly. that's why Express even has Route so you could build them independently and then mount to each other in any way you like.

a lot of porjects in node don't even have a thing called controllers, they just do very DRY routes and then there are models and if things get out of hands then there are also either services or utils depending on the nature of a project. a normal structure would be like so:

src/
  middleware/
  models/
  services/
  routes/
  app.js - the app without HTTP server
  server.js - the HTTP server that boots the app
test/
  middleware/
  models/
  services/
  routes/

Your structure seems heavily skewed towards controllers/services. You kind of have empty rudimentary models and routes, but then ended up with a bucked labeled controllers where you dump pretty much everything else. That's not gonna scale that well.

Also if you have noticed it led you to pretty chunky controller tests that also test routes as well. and the result you don't have routes tests at all. In a normal scenario, your routes tests would be your integration tests and middleware / models / services will contain unit tests for critical parts

#37

Another discussion that kinda gets lost in this, is using fluent assertions. I love em. Yeah assert does the job, but it's more taxing to read down a page and find what's being asserted - you've gotta read through each line of code to get the context you need to understand why assert(obj1 == result). Easier understood as obj.must.be.true().
I find the exact opposite. I like that assert has a nice simple syntax that's unambiguous. In your example it would actually be assert(obj1) as equivalent to obj1.must.be.true()

Or potentially:

assert.deepEqual(obj1, result) and obj1.must.eq(result)

Either way, they both capture exactly the same amount of information in a single line, requiring the same amount of spelunking through the state of the rest of the function.

Assert, however, does it without monkeypatching the global prototype of All The Things. It's also part of stdlib, so it's one less (or in the case of must, 8 less, http://npm.anvaka.com/#/view/2d/must) vector by which we can be left-padded.

I'd like to start a discussion on the naming of things in redbeard.

Right now, I believe there is confusion around the current Route, Controller, Model.

In the current form,
Route -> The code that defines the path to a route and The code that takes req and res, retrieves data, and passes it to res.send()
Controller -> The code that queries rethinkdb and returns data'
Model -> The schema of the models stored in rethinkdb

Perhaps an alternative naming convention, to follow that of rails, could be:
Route -> The code that defines the path to a route, and the controller action to use
Controller -> A group of functions (actions) that take a req and res, retrieves data, and passes it to res.send()
Model -> The code that queries rethinkdb and returns data
Schema -> The schema of the models stored in rethinkdb.

Replacing would look something like this, old (would now be called) -> new.
? -> Route (importing controllers instead of defining them as anonymous functions)
Route -> Controller
Controller -> Model
Model -> Schema

First steps:

• Sign up
• Sign in / Sign out with token auth
• Password reset

Extras

• Middleware that authenticates auth token and returns current user to controller or handles request with a 403 if authentication fails
• Emails for sign up and pw reset

While working on the new naming conventions branch, redbeard controller X doesn't feel right anymore. The command spits out routes, a controller, a model and a schema.

I suggest we switch the name to resource. This, in the future, allows us to reserve controller for more fine grain generation of files, eg: controller spits out just the controller, model spits out just the model, etc.

There is a test for this functionality, see websockets should orderBy asc if asked

But the test is invalid as the result from sockets is sorted before the comparison is made. This needs removing from the test...

const reordered = _.clone(results).sort((a, b) => {
  if (a.id > b.id) return 1;
  if (a.id < b.id) return -1;
  return 0;
});

Then we need to track down why sorting is not working as expected.

Moving this from redbeard-admin here

I feel like JSCS was a good alternative to JSHint a year or two ago. But those days I think most projects standardized on a babel + eslint stack.

they kind of both do the same thing, but you're trying to standardize the stack so I think it would be only fitting to use it in here as well. even for the sheer least surprise factor.

UPDATE jscs 3 is actually utterly broken jscs-dev/node-jscs#2219 they've already released several patch versions and the problem is still there. i had to fall back to 2.x for redbeard-admin in order to make this work. i think this adds to the story

When you type redbeard you get a gross unhandled exception rather than useful information about what to do next. It would be better to catch the invalid command and provide a help listing of available commands.

As per #48

I've noticed you do JWT handling manually in a custom middleware. did you consider https://www.npmjs.com/package/restify-jwt ? it's kind of the same thing as express-jwt

Did you know that if you'd use mocha and generators with async control flow, your controllers tests could be so much cleaner and easier to extend and maintain. here's an example

https://github.com/Prismatik/opensourcify/blob/master/test/routes/github_test.js

script that auto-runs tests with chokidar, both in the main project and in generated projects.

controller.mustache file is at the root level (inside the controller folder) but ends up in /controllers after the generator is run. Likewise for test, route, table, fixture and schema. These are all the files where they are named based off the model's plural name, but I think it would reduce confusion to put them in the corresponding folders even if it's just a single file in each.

change

/controllers
> controller.mustache
> test.mustache
> route.mustache

to this

/controllers
> /controllers/controller.mustache
> /tests/controllers/test.mustache
> /routes/route.mustache

When generating a relationship, the related property is incorrectly placed at the root of the schema. This should be placed under properties.

Example: redbeard controller wolf -r sheep generates:

{
  ...
  "properties": {
    "id": {
      "type": "string",
      "faker": "random.uuid"
    },
    "name": {
      "type": "string",
      "faker": "name.findName"
    }
  },
  "sheep": {
    "type": "string",
    "faker": "random.uuid",
    "pattern": "^[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12}$"
  },
  "links": [
    {
      "rel": "sheep",
      "href": "/sheep/{sheep}"
    }
  ],
  ...
}

Should be:

{
  ...
  "properties": {
    "id": {
      "type": "string",
      "faker": "random.uuid"
    },
    "name": {
      "type": "string",
      "faker": "name.findName"
    },
    "sheep": {
      "type": "string",
      "faker": "random.uuid",
      "pattern": "^[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12}$"
    },
  },
  "links": [
    {
      "rel": "sheep",
      "href": "/sheep/{sheep}"
    }
  ],
  ...
}

Currently when there's an issue with user input we throw an error with a friendly message. This has the intended effect of telling the user what was wrong and stopping execution, but it also fills their terminal with a stack trace that isn't really any use to them. It would be better to console.error the message and then process.exit(1)

i've noticed there is a whole bunch of scripts hanging in the root level of a project. that is a pretty crowded place, i think they should go into a bin/ folder or something

I've noticed when we generate a project the test files don't have a _test.js extension. That is a pain in the neck in an editor. you'll end up with two tabs both labeled blah.js and you won't know which one of them is test and which is code

I keep forgetting to write this ticket. I'm wondering what drove you guys to switch from express to restify? It seems like it is modeled after express 2.0. It does pretty much all the same as express minus a huge number of features.

I find their argument about express being focused on views is a bit dated. You could kind stretch it this way with express 2.x, but express 4 is as much decoupled from views as it gets. html rendering is just a piece of middleware there, like everything else, you don't have to install it if you don't want to. the world moved on with express, and restify kind of stuck in 2012.

For example, it pushes you to write routes like this:

module.exports = function(app) {
  app.use("/smth/smth", (req, res) => {
  });
};

There are plenty of articles online that suggest to do it this way, but that is because the way we did it back in the express 2.0 times. it was considered an anti-pattern for a long time now. basically because it breaks the abstraction level and creates coupling between an app and the routing. expressjs introduced a Router long time ago to solve this exact problem.

Here is an example how it is done in a modern express app. routes a recomposed of other routes and provide a single middleware that then mounted into an app.

https://github.com/Prismatik/opensourcify/blob/master/src/routes/index.js

Either way, I strongly suggest you reconsider the switch. In my opinion you won't win anything by marginalizing yourself from the mainstream. You just lock yourself out from new features and endless options of community built middleware

As per #62

@davidbanham This todo..

Potentially use a before/after/each plugin for Tape

Before/after is fairly trivial:

function beforeAfter(description, cb) {
  test(description, (t) => {
    // before promise
    someKindaPromise.then(thing => {
      cb(t).then(() => {
        // after promise
      });
    });
  });
}

Can easily split that up if both aren't needed, but the bones are there. I'm sure I was making a repo for some tape utils some time ago.. must've not finished it. I'll see if I can dig it up.

... or just switch to Mocha

I'd vote Mocha, but tape is pretty well stuck in there now..

Project names should only contain characters supported by rethinkdb as the name is used for the database name. Otherwise errors occur when starting the server...

Database name 'my-project' invalid (Use A-Za-z0-9_ only) in:\nr.dbCreate("my-project")\n

Same as in redbeard-admin, just to automate all the linting and testing before things go into git

The following function's returned function returns a promise, bit of a nitpick but unless i'm missing something I don't think it is necessary.

https://github.com/Prismatik/redbeard/blob/master/controller/route.mustache#L12

There's currently 3 separate snippets of code which are fairly similar, but with minor differences, that are used to loop through .mustache template files and then write the usable files.

We could standardise this creation process into a separate module.

The benefits I see are:

• Continuity and understanding: The creation code is the same everywhere, and will have one source of truth.
• Portability: If it's a separate module it could not only be used by redbeard but also by redbeard modules if they are only used by themselves eg. redbeard-react

https://www.npmjs.com/package/yargs#completioncmd-description-fn

Requires a little dancing with postinstall hooks, but would be an awesome user experience.

More good information here:

http://stackoverflow.com/questions/19990639/how-to-add-tab-completion-to-a-nodejs-cli-app

I've noticed that none of the auto-generated files have the "use strict"; marker in them. as we don't use any precompilers for the source code in here, this might lead to very annoying issues with block scope variables now and then. this can be avoided

I don't think migrations should be a part of the base template. A lot of projects, especially in a microservices oriented architecture don't need migrations. So, I recon it should be a separate thing.

Also, do we really want a home grown migrations engine? It's a pretty delicate area, proper fail overs, rollbacks, and support of transactions is a must. I'd feel much easier about running migrations, especially in production, if we were using a battle tested solution.

Moreover, migrations solution for an SQL and NoSQL dbs might be quite different in their nature and would benefit from using context specific packages. I don't think the one size fits all is the best approach here

Test case 33 and 34 sometimes fail:

not ok 33 should be equivalent
  ---
    operator: deepEqual
    expected: |-
      { id: '33fedbea-5ffb-40ff-b8e5-25c014524f19', name: 'esse' }
    actual: |-
      { id: '35718d99-f3cf-4722-9a6c-0a5877db05bc', name: 'esse' }
    at: Socket.<anonymous> (/Users/nicholas/work/pris/redbeard/redbeard_tests/Savory_Pheasant/tests/controllers/Sudden_Wombat.js:324:11)
  ...
not ok 34 should be equivalent
  ---
    operator: deepEqual
    expected: |-
      { id: '33fedbea-5ffb-40ff-b8e5-25c014524f19', name: 'esse' }
    actual: |-
      { id: 'd439cdec-c54b-4e74-bce1-ae0b40d1c3c0', name: 'esse' }
    at: Socket.<anonymous> (/Users/nicholas/work/pris/redbeard/redbeard_tests/Savory_Pheasant/tests/controllers/Sudden_Wombat.js:324:11)

CORS middleware already exists as part of /base - is it needed as a seperate generator?

I've noticed this at other prismatik projects as well, the folders structure is a bit off the beam. It basically looks somewhat like this:

env/ <- configs
lib/ <- more configs
routes/
src/models
tests/

Imho we could improve coherence if we follow a structure like this one:

bin/
  ... project specific scripts if needed
config/
  .. all configs
  index.js <- exports all configs

src/
  middleware/
    ... all middleware
  models/
    ... models
    index.js <- exports all models
  services/
    ... contains various business logic services
  routes/
    ... route handlers (one per resource)
    index.js has all the routes mounted together into one middleware
  app.js <- the app (without the server) just middleware routes bolted together
  server.js <- the HTTP server that boots the `app` on `PORT`
test/
  middleare/
     ... unit tests
  models/
    ... unit tests
  services/
    ... unit tests
  routes/
    ... integration tests for the app
  fixtures/
    ... all the fixtures
  helper.js <- the test env setup and configs/patches

I've seen a structure like that in many projects and scales very well. it leaves space for all sorts of things and you mostly know what goes where.

It is also very easy to navigate and match code with tests

would be great if the base generator would generate a .gitignore file with basic things like

.DS_Store
node_modules
npm-debug*
tmp
logs

maybe it could even init the repo and checkin the first commit ༼ つ ◕_◕ ༽つ initializing the redbeard

I was kind of wondering why you use a low level rethinkdb connector here? i recon an actual ORM like thinky would be a better option for an actual real life application. It handles relations, indexes, connections pool and so on. and it's quite widely adopted among rethinkdb users

I've noticed that you use blue-tape as the default testing framework. why not mocha? it's a more widely used solution by a huge margin with a wastly more options for extensions. and it supports promises out of the box too. not to say that it also supports immensely useful things like global before/after hooks and spits out better BDD specs

yargs seems pretty good, but commander is so much better.

it has a nice and clean declarative API for the CLIs, it automatically builds great help pages, supports sub-commands, handles input validation and so on. and it's a goto tool for many devs when they want to build a CLI in node

Another thing regarding testing. I've noticed that you use node-fetch for hitting your API in tests. Which is great but not nearly as safe and performant as mounting an app in the same process and hitting it through an internal socket as superagent does.

Also, not to brag, but i've built a thing called doubleagent which is kind of like superagent but is Promise based through and through and specifically tuned for testing JSON APIs (meaning it does JSON data serialization/deserialization and so on)

and it also decouples the JSON APIs testing from the assertion libraries. so, unlike superagent you can use it with whatevers and feel good about your day

There's a link to what "Clayton's" means, but no explanation about why the project is called redbeard. Let's add one! Should probably remove the Clayton's link too... @davidbanham this one's probably for you?

The idea of keeping all code within the controller was great initially, but it's gotten out of hand. Let's see what we can pare out of the controller template into a controller helper lib.

The first 65 lines of defined functions are an easy win.

There's also the potential to extract some of the code within the exported functions. We want to tread a fine line there between reducing noise, yet retaining flexibility to insert resource-specific business logic.