dolphin-emu / sadm Goto Github PK

Currently dolphin uses the 32bit toolchain to make 64bit binaries. The main advantages of using the 64bit toolchain is just that it can use more memory. Someone should experiment on the buildbot and see if it helps build times.

You can tell msbuild to use 64bit toolchain with /p:PreferredToolArchitecture=x64.

Is this builder really helpful?
AFAIK The only differences from normal build are that AUTOUPDATE is not defined and STEAM is defined.
STEAM macro is never used in the source code (appears to only be used in some cmake stuff). Probably this should be reworked and not exposed as a macro, anyway.

cc @OatmealDome

storesymbols.bat calls

"C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x64\symstore.exe"

Should be 8.1 version instead of 8.0 (it shouldn't matter, but still better form...).

Currently the change submission path via the Buildbot webhook API is not properly authenticated, and this could be an abusable flaw. Once Buildbot is moved to altair and modernized, add a new webhook parser plugin which supports some kind of HMAC signature, or at the very least something like HTTP basic auth.

(I think we might actually be able to do HTTP basic auth already as a stopgap, but haven't tried.)

Replace with some new code in central to send release changes via the webhook API (instead of the PB API which changes currently uses). Main complexity is that we need to have a local repo clone to run git describe and to find what branch commits are on (replicating the basic features of the git-what-branch Perl script we're currently using).

• central Git repo management
• git-what-branch basic reimplementation
• Wiki client to update the latest build revision there (currently broken since the mediawiki upgrade)
• Figure out some central API to trigger build re-submission

can we schedule the next windows build to occur after other build is done (e.g. while test may be occurring)?
debug test run takes a long time and it's annoying.

Should be fairly easy using the Nix role I wrote for the FifoCI radeon worker I run on my home infra: https://github.com/delroth/infra.delroth.net/blob/master/roles/fifoci-worker.nix

Currently FifoCI can only submit results by running a manage.py command on the server side, and only as the user running the backend service. The buildbot master happens to run on the same machine and uses sudo -u fifoci to run that script, but this has several issues:

1. It's ugly and terrible privsep.
2. It means we can't run the FifoCI backend and the Buildbot master on separate machines, which hinders the hive->altair migration.

Ideally this is replaced with an API call and a client that can be called by the Buildbot (either as a Python library, or a separate binary -- separate binary might be easier for now).

This is very minor, but recently with the removal of DolphinWx.app from the generated .dmg file, the .app file for Dolphin is no longer capitalized.

To be consistent with how the Windows executable is generated it should probably be capitalized.

Causing some PRs to become green before all builders have finished their job when other build jobs happen concurrently (for example, release/wip builds).

• #169
• Move Buildbot config to its own repo and modernize (poetry, poetry2nix, etc.)
• Convert some misc helpers (manifest generation, website notifier) to libraries and/or properly packaged scripts.
• Add a standalone runner instead of using the "standard" buildbot startup script and sourcing the config.
• Add a Nix role to run the Buildbot.

It seems that the "cleaning cg" and "purging pdb" build steps no longer do anything, and should be removed...?

Sometimes, after a PR update, GitHub shows this:

Amending and force-pushing fixes it.

http://superuser.com/a/461934

Not sure if our situation defaults to enabling multithreading, would be great if buildbot runner could test :)
Might require using lzma2, which is better anyways.

9:12 PM <•Helios747> hey delroth, can you fix the expired cert on dolp.in links?
...
9:13 PM <•delroth> Helios747: yeah... I should do that
9:13 PM <•delroth> need to figure out what's the recommended option nowadays for fetching LE certs
9:13 PM <•Helios747> I just cronjob it myself
9:14 PM <•Helios747> ubuntu has "certbot" which I throw into a weekly cronjob
...
9:14 PM <•delroth> yeah but certbot is annoying with how it changes your nginx configs
9:15 PM <•Helios747> Doesn't it have a "standalone" switch?
9:15 PM <•Helios747> which just spins up a python webserver
9:15 PM <•delroth> maybe
9:15 PM <•delroth> anyway, we've got a wildcard cert for *.dolphin-emu.org anyway, so it's really only dolp.in
9:15 PM <•delroth> I'll just do a DNS verification or something

You don't want to use standalone, you want to use the webroot option plus certonly. Webroot (for http-01) just drops the files in a folder you specify, I use /tmp/letsencrypt then tell the web server to serve anything matching /.well-known/acme-challenge/ out of /tmp/letsencrypt/.well-known/acme-challenge.

certonly doesn't attempt to install the certificate; instead, provide an "install hook" that tells nginx to reload the certs.

Getting this warning when running colmena apply.

• #169
• Modernize the FifoCI Git repo: poetry, poetry2nix, etc. without breaking the workers
• Write new role (should be fairly trivial)

add /p:RunUnitTests=true

I'm trying to automate beta releases for the flathub build.
Currently it downloads a specified commit from the dolphin git repo that's updated manually whenever there's a new blog post.
The tool to automate this process, flatpak-external-data-checker, requires either a branch or tag to update the commit.

I've tried pulling the commit hash from https://dolphin-emu.org/update/latest/beta and downloading the source from https://github.com/dolphin-emu/dolphin/archive/{commit}.zip but as the repo uses submodules this fails to build.

Would it be possible to get any one of these:

• A tag per release: 5.0-16793
• A branch that follows beta releases: beta
• A tarball (or 7z) artefact that includes submodules (this function but without the build step)

It seems like Buildbot is building Windows x64 (and possibly more) releases twice and racing to replace the released artifact. If a user or automated system downloads the first build artifact and compares it to a user who downloaded it a few minutes later, the two files differ.

An example of the duplicate builds:
Original: https://dolphin.ci/#/builders/6/builds/2486
Duplicate: https://dolphin.ci/#/builders/6/builds/2488
The only difference I can see between the builds (besides timestamp) is that the second build only marks @delroth as "Responsible", while the first build has 4 users marked.

This causes issues with package managers like Scoop, which download build artifacts and validate their hash. The automated commit Calinou/scoop-games@15cf5b5 claims the artifact from the duplicated build 18271 has SHA256 e0cc4e6e4da497736b2e0ca6088700ab037848866083d851b984b012151da185, when the file hosted on the website as "build 18271" has hash 738d372ff81527a7c37a58aa62d9abc2efc68cfb7b8a689e5081829857d08210.

The Scoop code that computes the hash is located here: https://github.com/ScoopInstaller/Scoop/blob/f93028001fbe5c78cc41f59e3814d2ac8e595724/lib/autoupdate.ps1#L277-L278, and the "Computed hash:" message is printed in this log (use the log search instead of Ctrl+F): https://github.com/Calinou/scoop-games/actions/runs/3954168346/jobs/6771222872