Should we add metrics to cold start? about benchmarks

[RafaelGSS]

Note that cold start is not such a critical thing anymore unless your application has extremely low traffic

I Agree. Although, I believe that the more we expose the metrics of Fastify more confident will be the consumers.

The relevant metric to track is the time between starting the Node.js process.

Good! Perhaps this metric can be the first one to create. @Eomm the fastify/fastify#2195 get this metric already? We can continue on that PR.

[RafaelGSS]

Would it mean without endpoints?

Yes, I mean, maybe increase the number of routes until reaching some limit and document it or to understand that each route can increase the startup time in X.

[mcollina]

I'm generically +1 to add that benchmark. Note that cold start is not such a critical thing anymore unless your application has extremely low traffic. (Note that on AWS the process is reused and it allows to make https://github.com/fastify/aws-lambda-fastify more performant than the express equivalent under traffic).

The relevant metric to track is the time between starting the Node.js process, calling .listen() and then closing the server as soon as it's done listening.

---

As far as Fastify goes, fastify/fastify#1598 sums it up. Our main bottleneck is in the Ajv dependency tree - TL;DR essentially executing a really long regexp we are not matching.

[Eomm]

I started something like this:
fastify/fastify#2195

To check if this (empty) repo can speed up the start!
https://github.com/Eomm/fastify-fast-startup

I would glad to work on this with you 👍

[Eomm]

The hard things is that the schema compiling is the slower part, so we need to define a set of use cases.

Because if we don't use schemas, for example, we are ignoring that component

[RafaelGSS]

We can do both actually.

I thinking about:

• Benchmark of raw startup
• Benchmark startup with schemas
• Benchmark startup with X plugins

I don't know if expose the memory consumption when we use a lot of schemas is a good metric though. What do you think?

[Eomm]

Benchmark of raw startup

Would it mean without endpoints? - maybe we could do a poll on Twitter to define a good tradeoff for a "typical fastify application"
For example, when I drafted that code I thought to try the start with 1, 100 and 1000 fake endpoint

expose the memory consumption

I think it would be not because a higher memory consumption :

• should not scare the user to use schemas
• it depends too much on the implementation (aka if the user reuse the schemas in the routes configuration)

For sure we could measure it to write down some "best practice"

---

Usually, I run in prod fastify applications with:

• between 10/20 routes with:
  • 3/4 hooks
  • add some fastify decorator
  • add some request decorator
  • header schema (high reuse)
  • body schema (quite always unique)
  • query string schema (high reuse)
  • response schema that has till 4 levels of nesting object

[Eomm]

Ok, would you like to work on a branch in this repo?
I would focus on creating a json schema fake generator

(I don't have write access here, so I should do PRs to your branch tho)

[RafaelGSS]

Ok, would you like to work on a branch in this repo?

Ok. How fastify/fastify#2195 is related to the work above? There is something to reuse?

[Eomm]

There is something to reuse?

I think so: the structure with benchmark.js (I choose it because it is quite easy to use and it is bullet proof) and the concept to generate routes
For sure feel free to start a fresh one if you know a better option!

[RafaelGSS]

Closing since the improvements is being discussed in the PR above.