howprogrammingworks / nodejsstarterkit Goto Github PK

See example: https://travis-ci.org/github/HowProgrammingWorks/NodejsStarterKit/builds/677618387

• Configuration parameters: { size: Number, timeout: Number }
• Wait in queue implementation (N requests)
• Error message on queue overflow
• Integrate with execution timeout #15
• Add to feature list

Note: execution timeout and queue timeout are not the same timeouts

Don't forget to test queue in end-to-end tests #11

• async select(table, fields, where)
• async insert(table, record)
• async update(table, delta, where)
• async delete(table, where)

Thanks to @belochub who reminded me this case from our previous implementation

• Static files
• API handlers

We need to mark methods available without auth and requires auth

0. Load core layer modules
1. Load internal node.js dependencies
2. Load external dependencies (installed from npm)
3. Create Application instance
4. Create Config instance
5. Create Logger instance and wait for open event
6. Load lib layer modules
7. Connect to database
8. Load domain layer, api methods and init scripts
9. Open server ports for listening

We can prepare N sandboxes in pool to speed up session starting

TLS certificate autogeneration with https://letsencrypt.org/

To prevent critical exceptions in main thread we need to minimize code execution in it.

Use crypto.scrypt and implement hashing in /api/registerUser.js and /api/signIn.js

If getting 'Semaphore timeout' exception in those 2 methods(Client.apiws and Client.api) we don't reach semaphore.leave. Therefore request which was added to the queue, it wouldn't be removed from the queue. And soon the queue will be busy with tasks with an expired timer

  async api() {
    const { semaphore } = this.application.server;
    await semaphore.enter();
    // ....
  /* Lets assume, we getting 'Simaphore timeout' and this error won't be handled by this method
  So we won't be able to reach semaphore.leave()
  */    
    try {
    // ...
      const proc = this.application.runScript(method, sandbox);
      const result = await proc(context)(args);
   // ...
    } catch (err) {
      this.application.logger.error(err.stack);
      this.error(this.res, err.message === 'Not found' ? 404 : 500);
    }
    semaphore.leave();
  }
}

Load automatically /domain/*.js

• Unittests
• API tests
• End-to-end tests

Support logging from sandboxes with console interface wrapped and injected.
No logs rotation support.

Dependency loader: /lib/dependencies.js

If file is not already compressed like gif, jpeg, zip, etc... we will load files and hold them after gz in shared memory.

After session started all requests should be sandboxed in a session sandbox.
Now we have shared sandbox for all API methods in application, see: #3
This shared sandbox will be used just for requests without sessions.

Object.freeze() does not freeze objects recursively, so after Object.freeze(lib) it is still possible to change object properties of lib.

Here are following options:

• Class Server and two subclasses: HttpServer and WsServer
• Rewrite without classes just functions
• Extract almost all functions but use outer classes to preserve interfaces
• Use strategy pattern
• Implement Client class

We need to pass application to all core modules not with aggregation (as we implemented this in v1) but with dependency injection, so we can use either global or sandbox context.

To see file names in api stack traces we need to pass file names to vm.Script.

Use ({ field: 'value' }); instead of module.exports = { field: 'value' };

Load static files into cache using SharedArrayBuffer

• Concurrency limit implementation
• Configuration parameters: { concurrency: Number }
• Integrate with Queue: #16
• Add to feature list

Remove utf8 in readFile for static files and backport this to v1.x

Make API transport-agnostic

In Windows, the fs.reader method gets paths like '\index.html' while the static() method waits '/index.html'

./lib/application.js

...
  async cacheDirectory(directoryPath) {
    const files = await fsp.readdir(directoryPath, { withFileTypes: true });
    for (const file of files) {
      const filePath = path.join(directoryPath, file.name);
      if (file.isDirectory()) await this.cacheDirectory(filePath);
      else await this.cacheFile(filePath);
    }
    fs.watch(directoryPath, (event, fileName) => {
      const filePath = path.join(directoryPath, fileName);
      this.cacheFile(filePath);
    });
  }
...

./lib/server.js

...
class Client {
  constructor(req, res, application) {
    this.req = req;
    this.res = res;
    this.application = application;
  }

  static() {
    const { url } = this.req;
    const filePath = url === '/' ? '/index.html' : url;
    const fileExt = path.extname(filePath).substring(1);
    const mimeType = MIME_TYPES[fileExt] || MIME_TYPES.html;
    this.res.writeHead(200, { 'Content-Type': mimeType });
    const data = this.application.cache.get(filePath);
    if (data) this.res.end(data);
    else this.error(404);
  }

...

• Configuration parameters: { timeout: Number }
• Timeout implementation
• Add to feature list

Don't forget to test timeout in end-to-end tests #11

Write logs to separate folder where each thread will create separate file with thread id in name.

From this repo: https://github.com/HowProgrammingWorks/ServiceWorker

We need everything we have in lib/server.js/Client.api in lib/server.js/apiws: sessions, context passing.

Serve static files and API separately, decompose /lib/server.js/handler

• Over http(s)
• Over ws(s)

All parts of this kit will be described as a contract to be implemented and replaced easily even by third-party developers. We will prepare tests for contracts and integration tests to be sure parts will work together.

Session sandboxes will not be frozen but default should

For following:

• Changing system code layer (domain code will autoreloaded)
• Update dependencies
• Restart hardware server

({ name1, name2 }) => { /* function implementation */ };

It's better to have different kinds of threads:

• serve static files with http(s)
• serve API with ws(s)
• other types...