Your guide provides following step:

To use the Github Actions provisioner you will need to create and install a new Github App with Self-hosted runners Read & Write permissions on the organization level and provide your config with the respective information.

While trying to create such an app I stumble upon 'Callback URL' field. It's required, but I can't fully understand what URL to provide.

The VirtualMachineView stops responding to keyboard input after the first reboot. While it shouldn't be an issue, as interactions should not be required outside of editor mode, it would be nice to fix it.

macOS 13.2.1
Cilicon 1.1.0

5 of 6 machines from our CI stopped working without any changes being made by our team today.

I've switched machines to editor mode and had enough time to compare network configs of working and non-working machines. There are differences.

Working config:

Non-working config:

All non-working configs have empty Router, 255.255.0.0``Mask and IPs starting from 169

What's interesting is that if I wait for a few minutes the VM will finally get valid network settings.

I've tried switching from DHCP to manual config and that helps with network problem, but I immediately do face another issue — something related to token:

I've found this exact exception in runner sources, but I couldn't figure out what's wrong and how to fix it.

I believe that token is valid, because one of CI machines is still working fine, using exact same token.
Anyway I've generated and provided new token with no luck.

I know that I'm using an outdated version of macOS and Cilicon — I've just had no time to install the update.

So at the end I have this questions:

1. What could have happened to network settings?
2. What is the problem with token?
3. What can I do to fix issues?

Placing a cilicon.yml in the home directory is unconventional for macOS. A more common approach is .cilicon.yml. Or use NSUserDefaults, which reads from ~/Library/Preferences

Unfortunately we due to the major changes in Cilicon 2.0 we had to break/remove support for GitLab runners.
Hoping that someone who uses GitLab could contribute the provisioner.
@ast3150 perhaps you could have a look?

First of all, thanks for building Cilicon!

Is there currently a way to share the host network?
We have connected the host machines to our VPN but the Vm seems to not share the host network and therefore isn't able to reach our Gitlab instance.

hey! i was looking to create a github action worker. i've made it through a few of the setup steps, but the error messages from the runner are really hard to decipher. is there a version i can run that has more verbose error messages/stack traces so i can understand what's wrong with my configuration?

also, i saw the medium post that was written about the release of this, are there any other resources that walk through setting up a github app + configuration for a runner using Cilicon?

Thanks to #38 it's now possible to define more advanced configuration options for GitLab Runners 🎉.
However I had some difficulties to get it to work:

1. Right now #38 isn't released while the Readme is already updated which is a little bit misleading (took me quite some time to figure out, why my configuration was ignored by the runner).
2. In the Readme it's not written that you have to pass your options as a multiline-argument omitting [[runners]] because this part is already present (see below). Instead it sounds a bit like you have to pass the path to a config.toml file.
   Relevant code in GitLabRunnerProvisioner.swift:

let copyConfigTomlCommand = """
mkdir -p ~/.gitlab-runner
rm -rf ~/.gitlab-runner/config.toml
cat <<'EOF' >> ~/.gitlab-runner/config.toml
[[runners]]
  url = "\(config.gitlabURL)"
  token = "\(config.runnerToken)"
  executor = "\(config.executor)"
  limit = \(config.maxNumberOfBuilds)
\(config.configToml ?? "")
EOF
exit 1
"""

Especially 2. should be addressed imho 🙂 ...

I except that my cilicon.yml config was invalid somehow, but failure message doesn't help at all to identify issue;

Propose to give at least filename or even line number which is invalid. This would help for learning curve.

Would be very useful to have command line interface for tool which allows to select for example cilicon.yml filename and it's path. This makes easier to integrate tool to deployment systems, such as salt-stack.

Add support for passing an advanced config to GitLab Runner provisioner

Motivation

Some features of the GitLab Runner are only available through an advanced config. Critically, this includes external runner caches. This is a desirable feature because it can vastly improve the build times for ephemeral build runners.

Solution

Introduce a new config parameter configToml in cilicon.yml, an optional multi-line argument. Users can pass any custom runner configuration here, including a cache configuration.

Change the runner command from gitlab-runner run-single to gitlab-runner run. This is required because run-single does not take a configuration file. Consequently, the configuration parameters are passed via the config.toml file to the runner, instead of being passed as parameters to the shell command.

First, let me start by saying thank you for this project! I've searched far and wide for a simple solution to this very problem, and had my first successful run using a self-hosted solution with a Mac Mini for my project

Since the newest release supports targeting the /repos endpoint and individual's projects rather than organizations, it might make sense to support the JIT config setup for Github Actions Runners. I ended up using a custom script provision using this endpoint along with ./run.sh --jitconfig {jitconfig} option.

So basically the command would look like:

jitconfig=$(curl -L \
  -X POST \
  -H "Accept: application/vnd.github+json" \
  -H "Authorization: Bearer <TOKEN with repo scope>" \
  -H "X-GitHub-Api-Version: 2022-11-28" \
  https://api.github.com/repos/OWNER/REPO/actions/runners/generate-jitconfig \
  -d '{"name":"{NAME_PARAM}-'$(uuidgen)'","runner_group_id":{RUNNER_GROUP},"labels":LABELS_ARRAY,"work_folder":{WORK_FOLDER}}' \
  | jq -r '.encoded_jit_config') && ~/actions-runner/run.sh --jitconfig $jitconfig

You already have access to most of these with the config options as they are currently. According to the docs, it also works with GitHub apps, so maybe it could even replace the current implementation (not confident on that).

I can try my hand at doing it myself, but wanted to get your opinion on its inclusion first.

Thanks again!

Hey.

I have a problem with Cilicon crashing on startup. It's probably related to a misconfiguration on my ec2-builder, but the crash makes it hard to tell what goes on.

I've attached it to this ticket, any idea where it's stopping? I've had a very similar config run fine on a local mac. This is with v2.1.0 from github releases.

crash.txt

edit I've built and uploaded cilicon to get symbols, and this is a bit more useful:

crash-sym.txt

Looks like it's crashing in the lease parser because /var/db/dhcpd_leases can't be found. On my builder I see

sudo ls /var/db/dhcpclient/leases
en4.plist       en7.plist

but not the /var/db/dhcpd_leases file.

Hey folks, GitLab Runners will always register and run with the setting untagged jobs set to true.
It would be nice if the configuration block could contain the attribute run_untaggedin the cilicon.yml to set this value during the GitLab Runner registration process [1] against the GitLab Server API[2].

Here is a draft example how the config block could look like:

provisioner:
  type: gitlab
  config:
     name: "cilicon-runner"
     url: "https://<gitlab-url>"
     registrationToken: "<registration-token>"
     run_untagged: false
     tagList: "macos-cilicon"

[1] https://github.com/traderepublic/Cilicon/blob/main/Cilicon/Provisioner/GitLab%20Runner/GitLabService.swift#L32-L34
[2] https://docs.gitlab.com/ee/api/runners.html#register-a-new-runner

Cilicon currently uses URLSession.bytes to download the OCI images.

Problems

This approach has some problems:

• Downloading files serially is rather slow and could be sped up by downloading chunks in parallel and combining the files at the end
• The memory usage is high because downloads are held in-memory
• If the download is interrupted at any moment, all chunks must be re-downloaded

Possible solutions

I've investigated some approaches how this could be handled differently. So far the most promising approach I've seen would involve switching to the AsyncHTTPClient library which offers FileDownloadDelegate. This would address the high memory load during downloads by streaming the download directly to the disc using SwiftNIO for non-blocking I/O.

Secondly, the downloads could be split into different files, called chunks. This would mean the downloader can check whether a certain chunk already exists on-disk before starting the download, which would mean that an interrupted download could be continued relatively easily by only re-downloading any chunks which are not yet completed.

The setupAndRunVirtualMachine() function in VMManager silently fails if an error in the Virtualization framework occurrs.

This is because the try syntax in a Task does not require the code to actually handle the error. (see https://forums.swift.org/t/task-initializer-with-throwing-closure-swallows-error/56066 )

My recommended solution is to run the entire method on @MainActor and just remove the Task altogether.

This can help users debug issues such as this, because important context is provided for debugging:

Hey team and @Marcocanc, it was hard to not recognize some of Tart's code in #46. For example, LayerV2Downloader seems like a copy/paste of Tart's DiskV2#pull:

https://github.com/cirruslabs/tart/blob/f4bc02d1751dd5ad447619515e7d73a9efc2d7e6/Sources/tart/OCI/Layerizer/DiskV2.swift#L39-L121

Here is a visual diff that shows similarities. All the variable names are the same, structure is the same, just some comments removed and different kind of error handling:

On behalf of Tart authors we are glad you found Tart code useful but unfortunately such usage is agains Tart's License.

In the cilicon.yml config file I specify the runner name, but that will change nothing in the generated RUNNER_NAME file.

I don't think this is an issue with Cilicon

Unless I've misunderstood the docs, it doesn't appear to be possible to create a vm with the Apple Hypervisor that has nested virtualisation enabled.
What this means is that things like android studio booting an emulator will fail, and provide errors like

HVF error: HV_UNSUPPORTED
qemu-system-aarch64: failed to initialize HVF: Invalid argument

Which google will rightly point you at code signing qemu and some other bits, until if you're like me you've spent hours falling down reddit+stackoverflow rabbit holes before it dawns on you whats going on, and what might be needed for it to be possible

🤦

Anyway, leaving this here to pay it forward hopefully with enough words that google finds it.

Please close issue unless you can see anyway to do nested virtualisation, vmware fusion and parallels appear to have something from google, but I'm not sure what workaround they've got for it.

Now to go build a amd64 box for android e2e testing :(

Thanks!

I got the vm up and running now. When the runner task is done it shutdown and starts up again as expected but then it will not start the github action runner script again so it's stuck booted up but not available to run anything.

Am I missing a setting that resolves this?

After running for the first time.

Is there any plan to release to homebrew ?

I was trying the sonoma xcode 15.01. image ( https://github.com/cirruslabs/macos-image-templates/pkgs/container/macos-sonoma-xcode/144771897?tag=15.0.1) but it seems they have application/vnd.cirruslabs.tart.disk.v2 for the image in their name and the VMManager is specifically filtering for application/vnd.cirruslabs.tart.disk.v1.

Im not sure why this is the case.

I notice the GitLab start.command runs a bunch of commands with sudo. This doesn't seem strictly necessary, it could just install the runner binary in ~/bin and run it from there, or something like that.

Is there a good reason why it's doing this with elevated privileges, or would you accept a patch that changes this behaviour so sudo is only needed for the shutdown at the end. (Which also has a bug in it… see #19 )

I'm trying to convert from v1.1.0 to v2.0 but I wasn't able to because the vm wouldn't start. I changed the yml file vmBundlePath to source, it tries to convert the VM to the new standard so I clicked yes, but then it is not able to connect and I get this error all the time:

I'm running this on a M2 Pro Mac mini. Ventura 13.2.1.

I was hoping to be able to use an M1 Mini I have with Github Actions instead of turning to something like BuildKite, and I notice that although Github only supports runner groups for organizations, self-hosted runners are technically supported for personal repositories without an organization.

I tried (just to see if by some chance it would work) setting the organization slug to my Github username, but no dice. Is there any way support for personal repositories could be added for adding self-hosted runners?

We have a bunch of Mac Studios for CI in our office at this moment but we can use it only for run a single job at a time right now due to flaky Xcode with parallel executions.

Can Cilicon start multiple VMs at the same time, which run independent from each others? This'd help us to ultilise the Mac Studios' resources better by parallelising multiple jobs in multiple VMS