cirruslabs / cirrus-ci-docs Goto Github PK
View Code? Open in Web Editor NEWDocumentation for Cirrus CI ๐
Home Page: https://cirrus-ci.org
License: MIT License
Documentation for Cirrus CI ๐
Home Page: https://cirrus-ci.org
License: MIT License
On the writing tasks page, the CIRRUS_WORKING_DIR docs are incomplete:
CIRRUS_WORKING_DIR | Working directory where
Also, is HOME one of the variables that is pre-defined?
Feature request to support AWS compute services as targets for scheduling tasks.
wss://api.cirrus-ci.com/ws and https://api.cirrus-ci.com/graphql
support for auth tokens would be nice also
Setting some sort of environment variable/yaml option to be able to choose whether scripts are executed with cmd
or powershell
would be great. Also the ps: MY_COMMAND
syntax expanding to powershell -Command "MY_COMMAND"
would be really useful.
Being able to make powershell the default is really powerful because since powershell is shipped with a lot of bash-style commands, it allows you streamline the configuration file for OSX, Linux, and Windows together, ie for example you can use $VAR
everywhere, cat MY_FILE
everywhere, ls
everywhere, etc.
Perhaps the CIRRUS_SHELL
environment variable might be the most intuitive thing to bind this to.
is this in the roadmap, and do you have a ETA ?
I currently use a different provider, but i prefer docker based builds which you have
I was chatting with @fkorotkov on Twitter about handling exceptional situations.
To provide some context, I build programs using a C++ toolchain, and am using a package manager called Conan, and a build system called CMake. Conan is responsible for invoking all the CMake commands, including the testing framework (called CTest). When using CTest, it will maintain a series of logs in the directory Testing/Temporary
, but only a pass/fail summary is output to screen.
CTest offers a flag for extracting failed tests' output, but I believe that exception handling is a powerful tool for a robust system, and that this would be a boon to Cirrus configurations. Ideally, what I'd like to be able to do is something akin to what's below. Note: this example might be asking for more than the proposed title.
script:
configure:
- mkdir build
- cd build
- conan install ..
- conan build .. --configure
build:
try:
- conan build .. --build
on_failure:
- echo "Compilation failed"
test:
try:
- conan build .. --test
on_failure:
- cat Testing/Temporary/LastTest.log
In order to encrypt a variable go to repository's settings page via clicking settings icon on a repository's main page (for example https://cirrus-ci.org/github/my-organization/my-repository) and follow instructions.
I didn't find any examples in the docs. Would be good to have build failure notifications (ideally with selectable addressee and selectable branches etc) like Travis.
Builds triggered after the creation of A PR does not provide the following environment variables:
This is because Cirrus CI considered any new builds triggered after the creation of a PR as a build from the branch.
In our development workflow, it's mandatory for us to know that we are working on a PR (i.e. we are relying on the 3 above environments variable)
The only option seems to cancel any new build from the branch and to consider them as PR build.
When I encrypt a value, I get something like ENCRYPTED[<long hex string>]
back that I can use in my script to reconstitute the secret and use it.
If I understand correctly, this long hex string is an internal identifier, and not the actual encoded secret.
This is a good security measure, but this also implies that the encrypted secret is stored somewhere at Cirrus. What happens if we would like to purge a (now unused) secret from Cirrus's database (to limit exposure to only those secrets actually in use, in case Cirrus is attacked, for instance). There doesn't appear to be a list of the encrypted IDs anywhere to delete specific ones.
Useful for building notifications based on build results
Feature request to support Azure compute services as targets for scheduling tasks.
Not sure if this is a bug, but it caught me out and seems counter-intuitive. (Also, not sure if this repo is the right place, it's not really a docs issue nor a website issue).
Given this config:
task:
env:
SHARD: tests
test_all_script:
- bin\cache\dart-sdk\bin\dart.exe -c dev\bots\test.dart
matrix:
- name: tests-windows
env:
SHARD: tests
- name: tool_tests-windows
env:
SHARD: tool_tests
The SHARD env variable is set to tests
for both builds, despite the matrix
. In this case it's redundant at the top level, however if you wanted to set an env variable for all items in the matrix except one, it'd be convenient to write it this way (and only "override" it in the matrix entry you need to).
It'd be useful to see percentages of when caches are hit/missed and also uploaded. A cache that is good today might get broken in future (eg. every build writes into that folder) and go unnoticed. It'd be great if this was called out somewhere ("Hey, this cache is missing 99% of the time!").
Hello @fkorotkov, I've another issue I want to address by asking if it is possible in anyway to delete failed task build logs from https://cirrus-ci.com? E.g. a task build carries sensitive information and data due to a incorrect/invalid .cirrus.yml
configuration setup.
Steps to reproduce:
Actual result:
Menu blinked and stayed the same.
Expected result:
Menu show show sub-items.
Environement:
Firefox 54 on Fedora 25
In the docs, the description of the depends_on
feature of the config file uses an example that has no names, only named tasks.
lint_task:
script: yarn run lint
test_task:
script: yarn run test
publish_task:
depends_on:
- test
- lint
script: yarn run publish
But it appears that there is another way to specify the depends_on
names.
task:
name: lint
script: yarn run lint
task:
name: test
script: yarn run test
task:
name: publish
depends_on:
- test
- lint
script: yarn run publish
And it's not really clear how to combine the two:
lint_task:
script: yarn run lint
matrix:
- name: lint_foo
env:
SOMETHING = foo
- name: lint_bar
env:
SOMETHING = bar
task:
name: test
script: yarn run test
task:
name: publish
depends_on:
- test
- lint_task
script: yarn run publish
Can I say something depends on just the lint_foo
task (it doesn't appear to work, at least)?
It appears that if I leave the task name as just task
, then it uses the name: for dependencies, but if I set it to lint_task
, then it uses lint
as the name. Is there any way to say something like depends_on: lint_task:lint_foo
?
(I'd also note that some YAML parsers don't like that there are multiple tags with the same name at the same level, but that's really academic: it seems to work fine as long as there are names)
On Travis and AppVeyor you can have tasks that are allowed to fail. This is great for things like running tests against nightly builds of dependencies (for ex. I run the Dart Code tests against dev versions of Dart/Flutter and VS Code, but since they're more fragile, I don't want them to report failures in pull request statuses):
https://github.com/Dart-Code/Dart-Code/blob/master/.travis.yml#L15-L26
I can't find anything similar here, but it's a great feature. You could also abort all of the "allowed failure" tasks if any of the "required" tasks fail which may save some resources (there's no point running the optional ones if the build is known to be bad). You could possibly even lower the priority of optional tasks within a project (eg. run the required tasks for all of my branches before starting the optional ones).
"Re-run all failed tasks" button on a build view can be helpful for rerunning multiple tasks.
Cron builds would allow to schedule the execution of a build on a specified branch, at fixed intervals, independently of whether any commits were pushed to the repository.
Configuring a cron build from cirrus-ci.com would require to define the interval (hourly, daily, weekly or a custom value) and the branch (by default the branch marked as "default" in GitHub should be used).
Cron builds send notifications as any other builds.
I'm trying to set up a VS Code extension on Cirrus with tests. It requires running xvfb
for VS Code to work. The furthest I've gotten is like this:
apt-get update
apt-get -y install code code-insiders libasound2 libxkbfile-dev pkg-config libsecret-1-dev libxss1 libgconf-2-4 dbus xvfb libgtk-3-0
export CXX="g++-4.8" CC="gcc-4.8" DISPLAY=:99.0
Xvfb &
However this step just hangs with no visible log streamed to the web UI (this may be related to a discussion we had elsewhere about background tasks?). I expected the &
to run it in the background and then continue, so that the other steps (for ex. running the tests) would run.
(I don't know if this is an appropriate repo for something like this, but I figured nobody else can answer this and it's better to have the answers publicly searchable in case others hit this in future than in emails! Let me know if there's a better place for it).
Extending the operators supported by only_if
to regexp matching would allow to check branch naming conventions, for example:
build_task:
only_if: $CIRRUS_BRANCH == 'feature/'*
I try gradle caching with reference to Examples - Cirrus CI
container:
image: cirrusci/flutter:latest
check_task:
gradle_cache:
folder: ~/.gradle/caches
check_script: gradle check
cleanup_before_cache_script:
- rm -rf ~/.gradle/caches/$GRADLE_VERSION/
- find ~/.gradle/caches/ -name "*.lock" -type f -delete
However, the following error has occurred.
find: '/home/cirrus/.gradle/caches': No such file or directory
Why find can not find directories? rm is no problem...
What is causing the error?
Maybe I'm alone who feel uncomfortable to read a white text on the black background but I'd like to share my experience with you. Please, consider adding classic, black-on-white, theme to your site (and especially to the documentation).
Part of my installation process requires me to set an environment variable from within a script. However, these variables do not seem to persist to the next script no matter how or where I export them.
On Unix I have tried:
export FOO=42
echo 'export FOO=42' > ~/.profile
echo 'export FOO=42' > ~/.bash_profile
echo 'export FOO=42' > ~/.bashrc
But upon starting up a new script, bash doesn't recognize $FOO
.
ie the following specification does not behave as I would expect (echo
ing 42
):
set_script:
- export FOO=42
- echo 'export FOO=42' > ~/.profile
- echo 'export FOO=42' > ~/.bash_profile
- echo 'export FOO=42' > ~/.bashrc
- echo $FOO
> 42
test_script:
- cat ~/.profile
> export FOO=42
- cat ~/.bash_profile
> export FOO=42
- cat ~/.bashrc
> export FOO=42
- echo $FOO
>
Similarly on Windows, SET
, SETX
, nor [Environment]::SetEnvironmentVariable
persist the environment variable past the script they were called from.
In particular, I am actually modifying the PATH
variable, which unfortunately causes problems for when it is parsed under env
since I am referring to existing variables with the proper batch and bash syntax. So yeah, it seems like I have set this by script but that isn't exactly going so well... ๐
I'm completely baffled by this and any help would be greatly appreciated.
I recently tried Cirrus-CI after coming from Circle-CI.
One feature I'm missing (or at least I have not found it) is storing artifacts (for reference: Circle-CI docs) after a build.
In my case I would like to store the build/reports
folder of a Gradle
build and be able to access it after the build through the web frontend. This would be especially usefull if my tests or static code analysis tools failed and the error is not printed to the console, since I have no way to see them without running the build locally.
Thanks in advance!
The ID of the pull request should be available as an environment variable, for example CIRRUS_PULL_REQUEST
. I suggest to not set the variable at all when no pull requests are open on the branch being built.
During my tests today, I came across this strange behaviour:
The task stays in the scheduled
state, and the created since
keeps increasing. Typically, I observed that scheduling takes around 1 minute on my project.
Both instances of the matrix build seem up.
Looking at cirrus-task-5717458405031936
serial port's logs:
Booting from Hard Disk 0...
2018/08/22 16:11:48 GCEWindowsAgent: GCE Agent Started (version 4.5.4@1)
2018/08/22 16:12:00 GCEMetadataScripts: Starting startup scripts (version 4.2.0@1).
2018/08/22 16:12:00 GCEMetadataScripts: Found windows-startup-script-ps1 in metadata.
2018/08/22 16:12:38 windows-startup-script-ps1: Exception calling "DownloadFile" with "2" argument(s): "Unable to connect to the remote server"
2018/08/22 16:12:38 windows-startup-script-ps1: At C:\Windows\TEMP\metadata-scripts760610747\windows-startup-script-ps1.ps1:4 char:1
2018/08/22 16:12:38 windows-startup-script-ps1: + (New-Object System.Net.WebClient).DownloadFile('https://api.cirrus-ci ...
2018/08/22 16:12:38 windows-startup-script-ps1: + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2018/08/22 16:12:38 windows-startup-script-ps1: + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
2018/08/22 16:12:38 windows-startup-script-ps1: + FullyQualifiedErrorId : WebException
2018/08/22 16:12:38 windows-startup-script-ps1:
2018/08/22 16:12:46 windows-startup-script-ps1: Start-Process : This command cannot be run due to the error: The system cannot find the file specified.
2018/08/22 16:12:46 windows-startup-script-ps1: At C:\Windows\TEMP\metadata-scripts760610747\windows-startup-script-ps1.ps1:5 char:1
2018/08/22 16:12:46 windows-startup-script-ps1: + Start-Process -FilePath 'agent.exe' -ArgumentList '-task-id', '571745 ...
2018/08/22 16:12:46 windows-startup-script-ps1: + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2018/08/22 16:12:46 windows-startup-script-ps1: + CategoryInfo : InvalidOperation: (:) [Start-Process], InvalidOperationException
2018/08/22 16:12:46 windows-startup-script-ps1: + FullyQualifiedErrorId : InvalidOperationException,Microsoft.PowerShell.Commands.StartProcessCommand
2018/08/22 16:12:46 windows-startup-script-ps1:
2018/08/22 16:12:46 GCEMetadataScripts: windows-startup-script-ps1 exit status 0
2018/08/22 16:12:46 GCEMetadataScripts: Finished running startup scripts.
This looks exactly like the kind of false-positives I am trying to run away from :) Would it be possible to let me bake this step into my VM image and so to avoid any kind of unreliable network downloads?
I'm going to leave this one running just to double-check that it will get killed by the 60 minutes timeout.
Hi,
The base branch targeted by the merge of a pull request should be available through an environment variable, for example CIRRUS_PR_BASE_BRANCH
. It should be unset or empty when build does not relate to a pull request.
The similar variable at Travis is TRAVIS_BRANCH
, which has a special meaning when build is initiated from a pull request (doc).
(contrary to #47 I double-checked that the variable does not exist yet!)
Hello, just discovered the following problem, that with a basic .cirrus.yml
setup containing the docker_builder:
keyword it is not possible to build windows-based docker images.
docker_builder:
only_if: $CIRRUS_BRANCH == 'master'
env:
DOCKER_USERNAME: ENCRYPTED[...]
DOCKER_PASSWORD: ENCRYPTED[...]
build_script:
- docker build --tag <...> .
login_script:
- docker login --username $DOCKER_USERNAME --password $DOCKER_PASSWORD
push_script:
- docker push <...>
Is there a keyword that I forgot or is it not supported yet that cirrus is able to build windows-based (and maybe mac/darwin-based as well?) docker images? Because the cirrus-ci build gives me the following error message:
Step 1/3 : FROM cirrusci/windowsservercore:2016
2016: Pulling from cirrusci/windowsservercore
// ...
image operating system "windows" cannot be used on this platform
MacOS X, specifically with retina displays, would be really cool to have
https://cirrus-ci.com/task/6596254498816000
This log shows failure related to 'compilationTraceFilePath' code that was NOT in associated commit:
flutter/flutter@393f927
The 'compilationTraceFilePath' code was introduced in a later commit:
flutter/flutter@2c0329a
Moreover, it seems that only one file (gradle.dart) from the later commit got pulled into the earlier run, otherwise this log would be showing more of similar errors.
P.S. I guess I could imagine a problem with the specific build task itself that it always pulls certain file(s) from HEAD regardless of what Cirrus tell it to? I don't know enough about Cirrus yet to tell if something like this would be possible.
Starting Windows machine seems to be slow no matter what. I observed timings between 2 to 9 minutes on Azure Container Instances. On Google Compute Engine, the time to boot the windows-1803-core
image seems to be around 3 minutes.
AppVeyor already tried to work on this issue according to its documentation on Hyper-V
It is a pool of pre-heated virtual machines, so, generally, builds start faster on Hyper-V cloud.
I see several options to implement such a feature:
N
additional ready containers / instancesThis isn't a blocker but more of a nice-to-have idea.
Failed to start: FAILED_PRECONDITION: Resource type 'Microsoft.ContainerInstance/containerGroups' container group quota '20' exceeded in region 'West US'. on windows tasks.
I would expect the jobs to queue up, as they did when I restarted them.
Hello, I've the following problem: With the setting depicted below the task build always results into a Failed in 43:43
state with the message Agent is not responding!
. My docker image (lets call it foo/bar
) is based on cirrusci/windowsservercore:2016
and is 7GB large. Any suggestions?
# ...
win_task:
depends_on: coverage
windows_container:
image: foo/bar
os_version: 2016
cpu: 4
# ...
I assumed that the agent cannot be started due to the huge docker image size, so I've tried a configuration parameter disk: ...
but this results in another Failed in 00:00
state with the message FAILED_PRECONDITION: Unknown field disk on 'windows_container' YAML node!
. Why is it possible to set the cpu:
parameter but not the memory:
and disk:
parameter just like with linux
containers?
# ...
win_task:
depends_on: coverage
windows_container:
image: foo/bar
os_version: 2016
cpu: 4
disk: 20
# ...
In my Travis scripts I had some commands that were conditional, like this:
if [[ "$OS" == "linux" ]]; then
apt-get -y install libsecret-1-dev libstdc++6 gcc-4.8 xvfb
export CXX="g++-4.8" CC="gcc-4.8" DISPLAY=:99.0;
sh -e Xvfb;
fi;
When I try something like this on Cirrus, I get:
/tmp/scriptsXXX.sh: 3: /tmp/scriptsXXX.sh: [[: not found
I see you have conditions on tasks, but in this case I just want it on a few scripts (currently I'm trying to share the same task on Linux + macOS rather than duplicating all the steps).
In this Cirrus run, the test failed, but the run succeeded.
Now, there is a bug in this script (it has Bash line with the "export" when it should do the Windows equivalent), but even if that wasn't there, I think it might have done the same thing.
I think the root of the problem is that Cirrus is prefixing a "call " for each line in the script, and in the .cirrus.yml
file, an extra blank line is getting added to the test_all_script, so Cirrus does call
on an empty line, which returns success, allowing the build to "succeed" when it didn't, really. I'm quite sure that test.dart is returning a 1 as an exit code.
task:
name: tool_tests-windows
env:
SHARD: tool_tests
windows_container:
image: cirrusci/windowsservercore:2016
os_version: 2016
cpu: 4
env:
CIRRUS_WORKING_DIR: "C:\\Windows\\Temp\\flutter sdk"
git_fetch_script: git fetch origin
setup_script:
- bin\flutter.bat config --no-analytics
- bin\flutter.bat update-packages
- git fetch origin master
test_all_script: |
export TEST_COMMIT_RANGE="$(git merge-base --fork-point FETCH_HEAD HEAD || git merge-base FETCH_HEAD HEAD)..HEAD"
bin\cache\dart-sdk\bin\dart.exe -c dev\bots\test.dart
Cirrus should probably either not add "call" to each line, or it should strip blank lines from the script.
https://cirrus-ci.org/guide/writing-tasks/#environment-variables
Host OS. Either
linux
,windows
ordarwiin
.
Maybe it should also be CIRRUS_OS
for consistency?
Connecting cirrus-ci to a European Google K8s cluster currently fails with message FAILED_PRECONDITION: java.lang.IllegalArgumentException: Not supported at the moment zone europe-west1-b!
. Only us-central1-a
is supported for the time being.
Supporting European regions would be much appreciated.
Here are some example failures:
https://cirrus-ci.com/task/5166072017715200
https://cirrus-ci.com/task/5659668516962304
https://cirrus-ci.com/task/6490844211183616
BTW, I'm trying to compile flutter/engine in a GCE instance with a lot of CPUs. Starting a GCE instance in 1 minute seems to me much faster than start a container instantly but then spend 4 minutes to download all the tools.
The Flutter build console uses a JSON API for Travis and an XML API for Appveyor to get the build results for those builds.
We'd like to do something similar for Cirrus: is such an API available, or do we just need to scrape the HTML?
The list of environment variables has OS now (yay!), but it has a typo (boo!).
The mac platform is identified as "darwiin" instead of the presumed actual value "darwin".
I'm trying to install yarn on your container so i can build my electron app for windows on a windows box.
I have this code which i believe should install yarn and choco
yarn_script:
- Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))
- choco install yarn -y
- refreshenv
but when i run a yarn command later down the process i get this error.
Unable to initialize device PRN
Cache miss for node_modules-284aa9424480e687f821eb01e365288f7030e2212707d46087d362cd6ada17d6! Populating...
yarn : The term 'yarn' is not recognized as the name of a cmdlet, function,
script file, or operable program. Check the spelling of the name, or if a path
was included, verify that the path is correct and try again.
At C:\Users\ContainerAdministrator\AppData\Local\Temp\scripts81855ad8681d0d86d1
e91e00167939cb.ps1:3 char:1
+ yarn install
+ ~~~~
+ CategoryInfo : ObjectNotFound: (yarn:String) [], ParentContains
ErrorRecordException
+ FullyQualifiedErrorId : CommandNotFoundException
Failed to execute populate script for node_modules cache!
Looking at the docs, there is an environment variable with a git hash for the currently commit ($CIRRUS_CHANGE_IN_REPO
- at least I think that's what it is: the doc string "Git SHA" doesn't really tell me anything about which SHA it is), but there doesn't appear to be one that represents the current commit on the source branch.
What I need to be able to do is run a git diff --name-only
on the range of commits that this Cirrus build represents: for instance, locally I would do git diff --name-only master..HEAD
.
I've tried exactly that, but the "master" branch doesn't appear to exist on Cirrus builds. I've tried using $CIRRUS_DEFAULT_BRANCH
instead of 'master', but that just resolves to 'master'.
Here's what I get when I use master..HEAD
(or just master
):
โฉ RUNNING: cd .; git diff -U0 --no-color --name-only master..HEAD -- *.dart *.cxx *.cpp *.cc *.c *.C *.h *.java *.mm *.m
fatal: bad revision 'master..HEAD'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.