entropic-dev / boltzmann Goto Github PK

The constraints on routes feature is neat, and we want it. It's a breaking change for people using the older way to version routes in fmw, so it will be a breaking change for us to include it. Major version bump. Chris thinks we can hide the fact that this changed, and do it as a minor version bump, which would be preferable. Recording this so we don't forget.

The templates middleware doesn't have config docs, though it is overed in the general website concepts documentation.

We have two that are BOLTZMANN:<feature> but others that don't add the framework name or don't follow the casing pattern. We should probably coalesce on boltzmann:<feature> for everything.

Note that this is not documentation, but closer to a manifesto, a set of reasons why this thing instead of all the other things. At that point we'll need a way to deploy such a thing, which @chrisdickinson probably has a good idea about.

You can enable esbuild support for websites with --esbuild=on but there are no docs about how to use this.

tested on node 14

It'd be handy to update files that are in place if they match the content hash of content we generated. For example, nodemon.json.

This would allow for the following:

tap.test('Frobnicators', _(async assert => {
  var frob
  tap.beforeEach(() => {
    frob = Frob.objects.create({...}) // recreated per-test in this test handler
  })

  tap.test('should frob evenly', _(async assert => {
    Frob.objects.filter({id: frob.id}).update({...}) // changes discarded before next test
  })
}))

Specifically we should track if we're in a test so we can use CHECKPOINT / ROLLBACK TO CHECKPOINT to persist some changes in the db. Redis... we'll continue to drop entirely between tests.

Use case: if a user accidentally removes the APP_MIDDLEWARE export without expecting to, we should give them a heads up. We could do this by always logging the total app-attached middleware count on startup in dev mode, and additionally grepping the file for APP_MIDDLEWARE if the count is zero and logging a big message with steps to remedy the situation.

I've been doing a lot of documentation updates between releases that would be useful to deploy. Our current docs build depends on a tag being pushed and the docs step running. If we pull the docs step into a different action with different triggers, however, we might be able to update the deployed docs on editing a release, say.

This would make the inevitable CORS debugging a little easier.

Bundling all the bugs, itchy sweaters, & feature requests we found up here:

• APP_MIDDLEWARE needs to be exported due to a regression (I believe! I had us carefully tip-toe around this one)
• npm init boltzmann --typescript ignores the typescript flag for some reason
• There's not really a clear story for moving from Boltzmann JS -> TS in a project (manually rename files, make sure we change require to import because it affects how types are resolved)
• The remnants of templating are itchy (void ``;, empty line comments a la // )
• Requests for interfaces in addition to types (IContext vs just Context); dig into this more
• Typescript npm run boltzmann:routes runscript needs TS_NODE_FILES=true.
• Request to scaffold at different dir levels (src/ or lib/) – putting everything at top-level is an itchy sweater

Specifically so the logger can have a request id associated with it automatically. This would enhance out-of-the-box logging with a pattern we'd like to support.

• automatically make it text/html, who are we kidding?
• should not be JSON.stringify'd if it's text/html! (right now we check if the return type of the middleware onion is a buffer in runserver, and if it's not we call JSON.stringify on whatever the result is.)

Document the livereload feature.

Use the information from routing, validation middleware, and possibly typescript return types[1] to generate Swagger YAML for ingestion into various tools.

The goal is to be able to feed Boltzmann-generated OpenAPI documents into various documentation generators & request tools (like Postman.) A stretch goal is that we can lint these API docs.

[1]: We could also expose a validate.returns({...jsonschema}) middleware that does response validation on the way out in dev mode for non-typescript projects? IDK exactly.

An earlier cut of #101 had a bunch of hooks on events like unhandledRejection which attempted to gracefully shut down tracing before an exit. However, there are a few problems with this:

1. Boltzmann today doesn't do anything like this - if we were to handle those cases for honeycomb, we would need to handle them for non-honeycomb cases too, and doing so would be a major-version API change
2. At least some of the events handle async actions poorly - we would need to research this and figure out what's realistically possible.

Our current approach for #101 is to only listen for the onExit event, which should fire on a non-error exit. This should get us graceful shutdown in the event of a service roll. If the app hard crashes we may lose some trace data, but if that's the case we have bigger problems which will surface themselves through on-call incidents. This approach is acceptable for now, but following up on this behavior later - in both honeycomb and non-honeycomb cases - could be worthwhile.

While having application-wide body parsers is sufficient for many use cases, it is useful on occasion to replace the body parsing behavior for a single handler (or subset of handlers wrapped by middleware.) To that end, we should expose some API for replacing the set of body parsers available to the Context object. It might be something as straightforward as recording the body parser array on the Context class & exposing a get/set API for it.

Document what the --staticfiles feature does and how to use it.

Things a human currently does as our release process:

• bump the version in Cargo.toml
• update links in the top-level README to the new version number (possibly other places?)
• update examples so they have the latest version of the code
• git commit the above
• git tag the above with the new version number
• push code & tags to trigger Github workflow builds of release artifacts & docs
• write release notes
• publish the release

It seems to me that everything except the last two steps are steps a robot could do for us, given initial input from a human of the desired new version number. With workflow changes, we could also automate a changelog eventually, though I might argue for a human writing useful release notes for another human.

A gap we noticed when working on #101 is that span attributes for honeycomb aren't appropriately documented. Adding them would be a good follow-up!

Consider writing an example for it.

"typescript build error since there's a postinstall instruction in api/package.json to build the api package and it looks like some version got upgraded for JWTs that is causing an error in boltzmann.ts at templates/boltzmann/middleware/oauth.ts
if you correct the line to
if (!([] as string[]).concat(decoded.aud || []).includes(clientId)) {
then it will build properly during the install"

We would like to add OpenTelemetry support for the honeycomb tracing feature in boltzmann. It's a bit of a bear - a lot of moving parts - so I wanted to capture a loose list of stuff I need to do to make this work.

• Ensure tests pass on latest
• Add Sdk#start init call/boilerplate near top of bundle
  • Peruse the boltzmann bundle generated today and identify a place to inject the calls - this is probably entrypoint.ts but the call has to happen "early" so there's some risk here
  • Find a good way to hook onto the Promise returned by Sdk#start - this is likely a module-scoped variable
  • Run tests to ensure this isn't exploding
  • Add a new test to ensure startup is behaving
• Do whatever the rough equivalent of startTrace is for otel in the honeycombTrace middleware - this is probably pre-populating the active context and creating a parent span
  • Extract metadata fields from request and response - the same fields are passed to the initialized trace in the beeline instrumentation, but they'll be added to the active context
  • Investigate trace parent related request headers and ensure they're plumbed in
  • Convince myself that the calls are being made after the SDK is started
  • Convince myself that the active Context is in a healthy state
  • Ensure current tests are passing
  • Add a new test that covers the new functionality
• Create/close a span in the honeycombMiddleSpans middleware
  • Create a child span, following the express auto-instrumentation example
  • Add the same metadata fields as beelines do today
  • Convince myself that the active Context is in a good state, etc
  • Ensure current tests are passing
  • Add a new test that covers the new functionality
• (TK) Develop an integration test plan
• (TK) Actually do the integration test plan
• (TK) Prepare for a release
• (TK) Actually do the release

The problem: We want to know when a package.json run script is ours to update at will without undoing deliberate changes made by people working on the service.

The solution: Prefix our scripts with a B=1 to set a most-likely-harmless env var. If we see this prefix on an update run, we manage that run script. If this string is missing, we leave the run script alone.

The problem caused by the solution: B=1 is not interpretable by Windows PowerShell and causes any invocation of a runscript with it to fail. (It seems fine in PowerShell on Mac. ?? )

We could use cross-env to continue setting an env var, but the point of decorating the run scripts isn't setting an env var; it's decorating the run-scripts so we can decide if we can edit them. It feels a little silly to add an entire unused dep for this.

We could find another no-op prelude to the runscripts, one that works cross-platform. echo && might work.

Or we could always update every run script we know about that starts with boltzman: and leave anything else alone if present. This is predictable behavior, at least.

Today, we're using the older structopt API, but the features got rolled into the recently-released clap v3. Upgrading is a chore we should do!

Per the honeycomb-beeline docs, the SDK will automatically pickup HONEYCOMB_WRITE_KEY and HONEYCOMB_DATASET (no IO). It will also pick up HONEYCOMB_API_HOST, used for proxies (like Refinery).

Would it be possible to use these names as the preferred/default source for these values?