Setup benchmarks using Codspeed.io about twisted

[itamarst]

Current status: it's failing to work with the twisted/twisted repository, waiting on response from support.

[adiroiban]

Thanks for looking into this.

I have approved the codspee permissions... not sure if that was an issue.

[adiroiban]

I have invited you to the Twisted admin team... not sure why you were not already there.

https://github.com/orgs/twisted/teams/twisted-org-owners/members

@glyph maybe we can define a process in which the membership of the Twisted Owners teams is reviewed every 3 months ... it can be a security issue if we give permissions to too many people.
Maybe the review can be done together with the PSF meetings... and for example, if a member of
the team has not contributed in the last 3 months we will remove it.
If they are active again, we can add them back.

The admin teams have full power, and they can configure a repo to bypass any security checks or brach protections

[itamarst]

OK it's setup, next to try some benchmarks.

A note about benchmark implementations: a bunch of factors add a lot of randomness, e.g. Python's hash randomization. So to reduce noise you need to run the benchmark multiple time with different Python interpreter instances each time.

[itamarst]

Apparently codspeed.io does not use fixed hardware, they use cachegrind (ala https://pythonspeed.com/articles/consistent-benchmarking-in-ci/). Which is probably fine for Python. And they just use a fixed hash seed, which is ... probably fine for CI at least.

[adiroiban]

Looking forward to the codspeed results.

There are also some cheap bare-metal providers 1x AMD Ryzen PRO 3600
6C / 12T - 3,6 GHz , 16 GB RAM, at 43 Euro per month ... I exect they are real bare-metal servers ... - https://www.scaleway.com/en/pricing/?tags=baremetal,available

If needed, I can get a real bare-metal Ryzen 7 4800U (ASUS Mini PC PN50) to run the benchmark tests