Improve `generate_test_keys.py` about mbedtls

[mpg]

While at it, please also make sure the freshness of tests/src/test_certs.h (which is similarly both generated and checked in), is tested on the CI.

[gilles-peskine-arm]

like tests/src/test_certs.h: the file is commited and there is no check that it's up-to-date

This shouldn't be the case.

please also make sure the freshness (…) is tested on the CI.

I would prefer to avoid having both files that are generated and files whose freshness is tested. Files that are generated are a lot more convenient because committing them requires extra steps in day-to-day development and causes more build conflicts. So I'd prefer to avoid reintroducting freshness testing.

[valeriosetti]

like tests/src/test_certs.h: the file is commited and there is no check that it's up-to-date

This shouldn't be the case.

Right, I just found that test_certs.h should be generated from tests/data_files/Makefile, so I think test_key.h should follow the same pattern.

please also make sure the freshness (…) is tested on the CI.

I would prefer to avoid having both files that are generated and files whose freshness is tested. Files that are generated are a lot more convenient because committing them requires extra steps in day-to-day development and causes more build conflicts. So I'd prefer to avoid reintroducting freshness testing.

Having these header files generated from a Makefile should allow to skip the check for "freshness", isn't it?
However I'm not sure tests/data_files/Makefile is ever called during the build. But I might be missing something...

[mpg]

I would prefer to avoid having both files that are generated and files whose freshness is tested.

Fully agreed, glad you said that. (I was afraid there was a reason why test_cert.h had to be treated in a special way, but you clarified on Slack that wasn't the case.) So, let's make them both (test_certs.h and the new test_key.h) generated.

Having these header files generated from a Makefile should allow to skip the check for "freshness", isn't it?
However I'm not sure tests/data_files/Makefile is ever called during the build. But I might be missing something...

Indeed, what we want it to have them generated during the build:

• in a Makefile other than tests/data_files/Makefile (as that one is not really part of the build system);
• and in CMake (as we support both build systems).

So, basically, this issue is about making the following changes with test_key.h:

1. Have external key files (commited, generation procedure documented in tests/data_files/Makefile) and then generate test_key.h from these files.
2. Make it generated as part of the build (make and CMake).

And #9015 is about doing just (2) for test_certs.h. IMO it might make sense to address both this issue and #9015 in a single PR, unless part 1 is too big, in which case a part for part 1, then another PR for part 2 and #9015.

[mpg]

Actually, once part 1 is done, the generation scripts for test_certs.h and test_key.h will start looking very similar and probably want to share more code. Actually would if make sense to have a single generation script and a single file tests/src/test_keys_certs.h? After all, test_certs.h has a few keys as well, so if everything is in a single file perhaps we could avoid some duplication (like, if we already have a test certs with an RSA-2048 private key, we don't really need an extra RSA-2048 private key for key testing, we can just re-use that one). Wdyt?

[valeriosetti]

I agree on both last comments (1 and 2). I will implement them