cirruslabs / cirrus-ci-docs Goto Github PK

On the writing tasks page, the CIRRUS_WORKING_DIR docs are incomplete:

CIRRUS_WORKING_DIR | Working directory where

Also, is HOME one of the variables that is pre-defined?

Feature request to support AWS compute services as targets for scheduling tasks.

wss://api.cirrus-ci.com/ws and https://api.cirrus-ci.com/graphql

support for auth tokens would be nice also

Setting some sort of environment variable/yaml option to be able to choose whether scripts are executed with cmd or powershell would be great. Also the ps: MY_COMMAND syntax expanding to powershell -Command "MY_COMMAND" would be really useful.

Being able to make powershell the default is really powerful because since powershell is shipped with a lot of bash-style commands, it allows you streamline the configuration file for OSX, Linux, and Windows together, ie for example you can use $VAR everywhere, cat MY_FILE everywhere, ls everywhere, etc.

Perhaps the CIRRUS_SHELL environment variable might be the most intuitive thing to bind this to.

is this in the roadmap, and do you have a ETA ?
I currently use a different provider, but i prefer docker based builds which you have

I was chatting with @fkorotkov on Twitter about handling exceptional situations.

To provide some context, I build programs using a C++ toolchain, and am using a package manager called Conan, and a build system called CMake. Conan is responsible for invoking all the CMake commands, including the testing framework (called CTest). When using CTest, it will maintain a series of logs in the directory Testing/Temporary, but only a pass/fail summary is output to screen.

CTest offers a flag for extracting failed tests' output, but I believe that exception handling is a powerful tool for a robust system, and that this would be a boon to Cirrus configurations. Ideally, what I'd like to be able to do is something akin to what's below. Note: this example might be asking for more than the proposed title.

script:
  configure:
    - mkdir build
    - cd build
    - conan install ..
    - conan build .. --configure
  build:
    try:
      - conan build .. --build
    on_failure:
      - echo "Compilation failed"
  test:
    try:
      - conan build .. --test
    on_failure:
      - cat Testing/Temporary/LastTest.log

In order to encrypt a variable go to repository's settings page via clicking settings icon on a repository's main page (for example https://cirrus-ci.org/github/my-organization/my-repository) and follow instructions.

https://cirrus-ci.org -> https://cirrus-ci.com

I didn't find any examples in the docs. Would be good to have build failure notifications (ideally with selectable addressee and selectable branches etc) like Travis.

Builds triggered after the creation of A PR does not provide the following environment variables:

• CIRRUS_BASE_BRANCH
• CIRRUS_BASE_SHA
• CIRRUS_PR

This is because Cirrus CI considered any new builds triggered after the creation of a PR as a build from the branch.

In our development workflow, it's mandatory for us to know that we are working on a PR (i.e. we are relying on the 3 above environments variable)

The only option seems to cancel any new build from the branch and to consider them as PR build.

When I encrypt a value, I get something like ENCRYPTED[<long hex string>] back that I can use in my script to reconstitute the secret and use it.

If I understand correctly, this long hex string is an internal identifier, and not the actual encoded secret.

This is a good security measure, but this also implies that the encrypted secret is stored somewhere at Cirrus. What happens if we would like to purge a (now unused) secret from Cirrus's database (to limit exposure to only those secrets actually in use, in case Cirrus is attacked, for instance). There doesn't appear to be a list of the encrypted IDs anywhere to delete specific ones.

Useful for building notifications based on build results

Feature request to support Azure compute services as targets for scheduling tasks.

https://cirrus-ci.com/github/yschimke/oksocial

https://cirrus-ci.com/build/5733636452122624

Not sure if this is a bug, but it caught me out and seems counter-intuitive. (Also, not sure if this repo is the right place, it's not really a docs issue nor a website issue).

Given this config:

task:
  env:
    SHARD: tests
  test_all_script:
    - bin\cache\dart-sdk\bin\dart.exe -c dev\bots\test.dart
  matrix:
    - name: tests-windows
      env:
        SHARD: tests
    - name: tool_tests-windows
      env:
        SHARD: tool_tests

The SHARD env variable is set to tests for both builds, despite the matrix. In this case it's redundant at the top level, however if you wanted to set an env variable for all items in the matrix except one, it'd be convenient to write it this way (and only "override" it in the matrix entry you need to).

It'd be useful to see percentages of when caches are hit/missed and also uploaded. A cache that is good today might get broken in future (eg. every build writes into that folder) and go unnoticed. It'd be great if this was called out somewhere ("Hey, this cache is missing 99% of the time!").

Hello @fkorotkov, I've another issue I want to address by asking if it is possible in anyway to delete failed task build logs from https://cirrus-ci.com? E.g. a task build carries sensitive information and data due to a incorrect/invalid .cirrus.yml configuration setup.

Steps to reproduce:

• open https://cirrus-ci.org in Firefox
• zoom a page to 175% (Ctrl+Mouse Wheel)
• in this mode right menu became invisible
• click on the picture at the left top corner
• menu will appear on the left side
• click on item "User Guide" or "About"

Actual result:
Menu blinked and stayed the same.

Expected result:
Menu show show sub-items.

Environement:
Firefox 54 on Fedora 25

In the docs, the description of the depends_on feature of the config file uses an example that has no names, only named tasks.

lint_task:
  script: yarn run lint

test_task:
  script: yarn run test

publish_task:
  depends_on: 
    - test
    - lint
  script: yarn run publish

But it appears that there is another way to specify the depends_on names.

task:
  name: lint
  script: yarn run lint

task:
  name: test
  script: yarn run test

task:
  name: publish
  depends_on: 
    - test
    - lint
  script: yarn run publish

And it's not really clear how to combine the two:

lint_task:
  script: yarn run lint
  matrix:
    - name: lint_foo
      env:
        SOMETHING = foo
    - name: lint_bar
      env:
        SOMETHING = bar

task:
  name: test
  script: yarn run test

task:
  name: publish
  depends_on: 
    - test
    - lint_task
  script: yarn run publish

Can I say something depends on just the lint_foo task (it doesn't appear to work, at least)?

It appears that if I leave the task name as just task, then it uses the name: for dependencies, but if I set it to lint_task, then it uses lint as the name. Is there any way to say something like depends_on: lint_task:lint_foo?

(I'd also note that some YAML parsers don't like that there are multiple tags with the same name at the same level, but that's really academic: it seems to work fine as long as there are names)

On Travis and AppVeyor you can have tasks that are allowed to fail. This is great for things like running tests against nightly builds of dependencies (for ex. I run the Dart Code tests against dev versions of Dart/Flutter and VS Code, but since they're more fragile, I don't want them to report failures in pull request statuses):

https://github.com/Dart-Code/Dart-Code/blob/master/.travis.yml#L15-L26

I can't find anything similar here, but it's a great feature. You could also abort all of the "allowed failure" tasks if any of the "required" tasks fail which may save some resources (there's no point running the optional ones if the build is known to be bad). You could possibly even lower the priority of optional tasks within a project (eg. run the required tasks for all of my branches before starting the optional ones).

"Re-run all failed tasks" button on a build view can be helpful for rerunning multiple tasks.

Cron builds would allow to schedule the execution of a build on a specified branch, at fixed intervals, independently of whether any commits were pushed to the repository.
Configuring a cron build from cirrus-ci.com would require to define the interval (hourly, daily, weekly or a custom value) and the branch (by default the branch marked as "default" in GitHub should be used).

Cron builds send notifications as any other builds.

I'm trying to set up a VS Code extension on Cirrus with tests. It requires running xvfb for VS Code to work. The furthest I've gotten is like this:

apt-get update
apt-get -y install code code-insiders libasound2 libxkbfile-dev pkg-config libsecret-1-dev libxss1 libgconf-2-4 dbus xvfb libgtk-3-0
export CXX="g++-4.8" CC="gcc-4.8" DISPLAY=:99.0
Xvfb &

However this step just hangs with no visible log streamed to the web UI (this may be related to a discussion we had elsewhere about background tasks?). I expected the & to run it in the background and then continue, so that the other steps (for ex. running the tests) would run.

(I don't know if this is an appropriate repo for something like this, but I figured nobody else can answer this and it's better to have the answers publicly searchable in case others hit this in future than in emails! Let me know if there's a better place for it).

https://cirrus-ci.com/build/5726749002301440

Seems to still download all dependencies

Extending the operators supported by only_if to regexp matching would allow to check branch naming conventions, for example:

build_task:
  only_if: $CIRRUS_BRANCH == 'feature/'*

I try gradle caching with reference to Examples - Cirrus CI

container:
  image: cirrusci/flutter:latest

check_task:
  gradle_cache:
    folder: ~/.gradle/caches
  check_script: gradle check
  cleanup_before_cache_script:
    - rm -rf ~/.gradle/caches/$GRADLE_VERSION/
    - find ~/.gradle/caches/ -name "*.lock" -type f -delete

However, the following error has occurred.

find: '/home/cirrus/.gradle/caches': No such file or directory

Why find can not find directories? rm is no problem...

What is causing the error?

Maybe I'm alone who feel uncomfortable to read a white text on the black background but I'd like to share my experience with you. Please, consider adding classic, black-on-white, theme to your site (and especially to the documentation).

Part of my installation process requires me to set an environment variable from within a script. However, these variables do not seem to persist to the next script no matter how or where I export them.

On Unix I have tried:

export FOO=42
echo 'export FOO=42' > ~/.profile
echo 'export FOO=42' > ~/.bash_profile
echo 'export FOO=42' > ~/.bashrc

But upon starting up a new script, bash doesn't recognize $FOO.

ie the following specification does not behave as I would expect (echoing 42):

set_script:
  - export FOO=42
  - echo 'export FOO=42' > ~/.profile
  - echo 'export FOO=42' > ~/.bash_profile
  - echo 'export FOO=42' > ~/.bashrc
  - echo $FOO
> 42
test_script:
  - cat ~/.profile
> export FOO=42
  - cat ~/.bash_profile
> export FOO=42
  - cat ~/.bashrc
> export FOO=42
  - echo $FOO
> 

Similarly on Windows, SET, SETX, nor [Environment]::SetEnvironmentVariable persist the environment variable past the script they were called from.

In particular, I am actually modifying the PATH variable, which unfortunately causes problems for when it is parsed under env since I am referring to existing variables with the proper batch and bash syntax. So yeah, it seems like I have set this by script but that isn't exactly going so well... 😅

I'm completely baffled by this and any help would be greatly appreciated.

I recently tried Cirrus-CI after coming from Circle-CI.
One feature I'm missing (or at least I have not found it) is storing artifacts (for reference: Circle-CI docs) after a build.

In my case I would like to store the build/reports folder of a Gradle build and be able to access it after the build through the web frontend. This would be especially usefull if my tests or static code analysis tools failed and the error is not printed to the console, since I have no way to see them without running the build locally.

Thanks in advance!

The ID of the pull request should be available as an environment variable, for example CIRRUS_PULL_REQUEST. I suggest to not set the variable at all when no pull requests are open on the branch being built.

During my tests today, I came across this strange behaviour:

The task stays in the scheduled state, and the created since keeps increasing. Typically, I observed that scheduling takes around 1 minute on my project.

So, digging into GCE:

Both instances of the matrix build seem up.

Looking at cirrus-task-5717458405031936 serial port's logs:

Booting from Hard Disk 0...
2018/08/22 16:11:48 GCEWindowsAgent: GCE Agent Started (version 4.5.4@1)
2018/08/22 16:12:00 GCEMetadataScripts: Starting startup scripts (version 4.2.0@1).
2018/08/22 16:12:00 GCEMetadataScripts: Found windows-startup-script-ps1 in metadata.
2018/08/22 16:12:38 windows-startup-script-ps1: Exception calling "DownloadFile" with "2" argument(s): "Unable to connect to the remote server"
2018/08/22 16:12:38 windows-startup-script-ps1: At C:\Windows\TEMP\metadata-scripts760610747\windows-startup-script-ps1.ps1:4 char:1
2018/08/22 16:12:38 windows-startup-script-ps1: + (New-Object System.Net.WebClient).DownloadFile('https://api.cirrus-ci ...
2018/08/22 16:12:38 windows-startup-script-ps1: + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2018/08/22 16:12:38 windows-startup-script-ps1:     + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
2018/08/22 16:12:38 windows-startup-script-ps1:     + FullyQualifiedErrorId : WebException
2018/08/22 16:12:38 windows-startup-script-ps1:  
2018/08/22 16:12:46 windows-startup-script-ps1: Start-Process : This command cannot be run due to the error: The system cannot find the file specified.
2018/08/22 16:12:46 windows-startup-script-ps1: At C:\Windows\TEMP\metadata-scripts760610747\windows-startup-script-ps1.ps1:5 char:1
2018/08/22 16:12:46 windows-startup-script-ps1: + Start-Process -FilePath 'agent.exe' -ArgumentList '-task-id', '571745 ...
2018/08/22 16:12:46 windows-startup-script-ps1: + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2018/08/22 16:12:46 windows-startup-script-ps1:     + CategoryInfo          : InvalidOperation: (:) [Start-Process], InvalidOperationException
2018/08/22 16:12:46 windows-startup-script-ps1:     + FullyQualifiedErrorId : InvalidOperationException,Microsoft.PowerShell.Commands.StartProcessCommand
2018/08/22 16:12:46 windows-startup-script-ps1:  
2018/08/22 16:12:46 GCEMetadataScripts: windows-startup-script-ps1 exit status 0
2018/08/22 16:12:46 GCEMetadataScripts: Finished running startup scripts.

This looks exactly like the kind of false-positives I am trying to run away from :) Would it be possible to let me bake this step into my VM image and so to avoid any kind of unreliable network downloads?

I'm going to leave this one running just to double-check that it will get killed by the 60 minutes timeout.

Hi,

The base branch targeted by the merge of a pull request should be available through an environment variable, for example CIRRUS_PR_BASE_BRANCH. It should be unset or empty when build does not relate to a pull request.

The similar variable at Travis is TRAVIS_BRANCH , which has a special meaning when build is initiated from a pull request (doc).

(contrary to #47 I double-checked that the variable does not exist yet!)

I use matrix to run tests on stable+dev versions of some dependencies, but I can't figure out how to include these in the name of the task so that it shows up nicely in the list.

Config looks like:

I tried just putting $CODE_VERSION etc. into the name, but it just showed up literally:

Hello, just discovered the following problem, that with a basic .cirrus.yml setup containing the docker_builder: keyword it is not possible to build windows-based docker images.

docker_builder:
  only_if: $CIRRUS_BRANCH == 'master'
  env:
    DOCKER_USERNAME:  ENCRYPTED[...]
    DOCKER_PASSWORD: ENCRYPTED[...]
  build_script:
    - docker build --tag <...> .
  login_script:
    - docker login --username $DOCKER_USERNAME --password $DOCKER_PASSWORD
  push_script:
    - docker push <...>

Is there a keyword that I forgot or is it not supported yet that cirrus is able to build windows-based (and maybe mac/darwin-based as well?) docker images? Because the cirrus-ci build gives me the following error message:

Step 1/3 : FROM cirrusci/windowsservercore:2016
2016: Pulling from cirrusci/windowsservercore
// ...
image operating system "windows" cannot be used on this platform

MacOS X, specifically with retina displays, would be really cool to have

https://cirrus-ci.com/task/6596254498816000

This log shows failure related to 'compilationTraceFilePath' code that was NOT in associated commit:
flutter/flutter@393f927

The 'compilationTraceFilePath' code was introduced in a later commit:
flutter/flutter@2c0329a

Moreover, it seems that only one file (gradle.dart) from the later commit got pulled into the earlier run, otherwise this log would be showing more of similar errors.

P.S. I guess I could imagine a problem with the specific build task itself that it always pulls certain file(s) from HEAD regardless of what Cirrus tell it to? I don't know enough about Cirrus yet to tell if something like this would be possible.

Starting Windows machine seems to be slow no matter what. I observed timings between 2 to 9 minutes on Azure Container Instances. On Google Compute Engine, the time to boot the windows-1803-core image seems to be around 3 minutes.

AppVeyor already tried to work on this issue according to its documentation on Hyper-V

It is a pool of pre-heated virtual machines, so, generally, builds start faster on Hyper-V cloud.

I see several options to implement such a feature:

1. Always maintain N additional ready containers / instances
2. Whenever pre-started VMs/containers run out, allocate 2x more
3. One of the above, but limited to a pre-configured working hours specified by the end-user
4. Based on historical data, attempt to predict future capacity needs based on some ML (for inspiration look at what Elasticsearch does)

This isn't a blocker but more of a nice-to-have idea.

Failed to start: FAILED_PRECONDITION: Resource type 'Microsoft.ContainerInstance/containerGroups' container group quota '20' exceeded in region 'West US'. on windows tasks.

I would expect the jobs to queue up, as they did when I restarted them.

Hello, I've the following problem: With the setting depicted below the task build always results into a Failed in 43:43 state with the message Agent is not responding!. My docker image (lets call it foo/bar) is based on cirrusci/windowsservercore:2016 and is 7GB large. Any suggestions?

# ...
win_task:
  depends_on: coverage
  windows_container:
    image: foo/bar
    os_version: 2016
    cpu: 4
# ...

I assumed that the agent cannot be started due to the huge docker image size, so I've tried a configuration parameter disk: ... but this results in another Failed in 00:00 state with the message FAILED_PRECONDITION: Unknown field disk on 'windows_container' YAML node!. Why is it possible to set the cpu: parameter but not the memory: and disk: parameter just like with linux containers?

# ...
win_task:
  depends_on: coverage
  windows_container:
    image: foo/bar
    os_version: 2016
    cpu: 4
    disk: 20
# ...

In my Travis scripts I had some commands that were conditional, like this:

if [[ "$OS" == "linux" ]]; then
  apt-get -y install libsecret-1-dev libstdc++6 gcc-4.8 xvfb
  export CXX="g++-4.8" CC="gcc-4.8" DISPLAY=:99.0;
  sh -e Xvfb;
fi;

When I try something like this on Cirrus, I get:

/tmp/scriptsXXX.sh: 3: /tmp/scriptsXXX.sh: [[: not found

I see you have conditions on tasks, but in this case I just want it on a few scripts (currently I'm trying to share the same task on Linux + macOS rather than duplicating all the steps).

In this Cirrus run, the test failed, but the run succeeded.

Now, there is a bug in this script (it has Bash line with the "export" when it should do the Windows equivalent), but even if that wasn't there, I think it might have done the same thing.

I think the root of the problem is that Cirrus is prefixing a "call " for each line in the script, and in the .cirrus.yml file, an extra blank line is getting added to the test_all_script, so Cirrus does call on an empty line, which returns success, allowing the build to "succeed" when it didn't, really. I'm quite sure that test.dart is returning a 1 as an exit code.

task:
  name: tool_tests-windows
  env:
    SHARD: tool_tests
  windows_container:
    image: cirrusci/windowsservercore:2016
    os_version: 2016
    cpu: 4
  env:
    CIRRUS_WORKING_DIR: "C:\\Windows\\Temp\\flutter sdk"
  git_fetch_script: git fetch origin
  setup_script:
    - bin\flutter.bat config --no-analytics
    - bin\flutter.bat update-packages
    - git fetch origin master
  test_all_script: |
    export TEST_COMMIT_RANGE="$(git merge-base --fork-point FETCH_HEAD HEAD || git merge-base FETCH_HEAD HEAD)..HEAD"
    bin\cache\dart-sdk\bin\dart.exe -c dev\bots\test.dart

Cirrus should probably either not add "call" to each line, or it should strip blank lines from the script.

https://cirrus-ci.org/guide/writing-tasks/#environment-variables

Host OS. Either linux, windows or darwiin.

Maybe it should also be CIRRUS_OS for consistency?

Connecting cirrus-ci to a European Google K8s cluster currently fails with message FAILED_PRECONDITION: java.lang.IllegalArgumentException: Not supported at the moment zone europe-west1-b!. Only us-central1-a is supported for the time being.

Supporting European regions would be much appreciated.

Here are some example failures:

https://cirrus-ci.com/task/5166072017715200

https://cirrus-ci.com/task/5659668516962304

https://cirrus-ci.com/task/6490844211183616

BTW, I'm trying to compile flutter/engine in a GCE instance with a lot of CPUs. Starting a GCE instance in 1 minute seems to me much faster than start a container instantly but then spend 4 minutes to download all the tools.

The Flutter build console uses a JSON API for Travis and an XML API for Appveyor to get the build results for those builds.

We'd like to do something similar for Cirrus: is such an API available, or do we just need to scrape the HTML?

The list of environment variables has OS now (yay!), but it has a typo (boo!).

The mac platform is identified as "darwiin" instead of the presumed actual value "darwin".

I'm trying to install yarn on your container so i can build my electron app for windows on a windows box.
I have this code which i believe should install yarn and choco

  yarn_script:
    - Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))
    - choco install yarn -y
    - refreshenv

but when i run a yarn command later down the process i get this error.

Unable to initialize device PRN
Cache miss for node_modules-284aa9424480e687f821eb01e365288f7030e2212707d46087d362cd6ada17d6! Populating...
yarn : The term 'yarn' is not recognized as the name of a cmdlet, function, 
script file, or operable program. Check the spelling of the name, or if a path 
was included, verify that the path is correct and try again.
At C:\Users\ContainerAdministrator\AppData\Local\Temp\scripts81855ad8681d0d86d1
e91e00167939cb.ps1:3 char:1
+ yarn install
+ ~~~~
    + CategoryInfo          : ObjectNotFound: (yarn:String) [], ParentContains 
   ErrorRecordException
    + FullyQualifiedErrorId : CommandNotFoundException
Failed to execute populate script for node_modules cache!

people like clicking things to confirm suspicions

e.g. let me start a new build from here

Looking at the docs, there is an environment variable with a git hash for the currently commit ($CIRRUS_CHANGE_IN_REPO - at least I think that's what it is: the doc string "Git SHA" doesn't really tell me anything about which SHA it is), but there doesn't appear to be one that represents the current commit on the source branch.

What I need to be able to do is run a git diff --name-only on the range of commits that this Cirrus build represents: for instance, locally I would do git diff --name-only master..HEAD.

I've tried exactly that, but the "master" branch doesn't appear to exist on Cirrus builds. I've tried using $CIRRUS_DEFAULT_BRANCH instead of 'master', but that just resolves to 'master'.

Here's what I get when I use master..HEAD (or just master):

⏩ RUNNING: cd .; git diff -U0 --no-color --name-only master..HEAD -- *.dart *.cxx *.cpp *.cc *.c *.C *.h *.java *.mm *.m
fatal: bad revision 'master..HEAD'