concourse / datadog-event-resource

Concourse: the continuous thing-doer.

Concourse is an automation system written in Go. It is most commonly used for CI/CD, and is built to scale to any kind of automation pipeline, from simple to complex.

Concourse is very opinionated about a few things: idempotency, immutability, declarative config, stateless workers, and reproducible builds.

The road to Concourse v10

Concourse v10 is the code name for a set of features which, when used in combination, will have a massive impact on Concourse's capabilities as a generic continuous thing-doer. These features, and how they interact, are described in detail in the Core roadmap: towards v10 and Re-inventing resource types blog posts. (These posts are slightly out of date, but they get the idea across.)

Notably, v10 will make Concourse not suck for multi-branch and/or pull-request driven workflows - examples of spatial change, where the set of things to automate grows and shrinks over time.

Because v10 is really an alias for a ton of separate features, there's a lot to keep track of - here's an overview:

Feature	RFC	Status
set_pipeline step	✔ #31	✔ v5.8.0 (experimental)
Var sources for creds	✔ #39	✔ v5.8.0 (experimental), TODO: #5813
Archiving pipelines	✔ #33	✔ v6.5.0
Instanced pipelines	✔ #34	✔ v7.0.0 (experimental)
Static across step	🚧 #29	✔ v6.5.0 (experimental)
Dynamic across step	🚧 #29	✔ v7.4.0 (experimental, not released yet)
Projects	🚧 #32	🙏 RFC needs feedback!
load_var step	✔ #27	✔ v6.0.0 (experimental)
get_var step	✔ #27	🚧 #5815 in progress!
Prototypes	✔ #37	⚠ Pending first use of protocol (any of the below)
run step	🚧 #37	⚠ Pending its own RFC, but feel free to experiment
Resource prototypes	✔ #38	🙏 #5870 looking for volunteers!
Var source prototypes		🚧 #6275 planned, may lead to RFC
Notifier prototypes	🚧 #28	⚠ RFC not ready

The Concourse team at VMware will be working on these features, however in the interest of growing a healthy community of contributors we would really appreciate any volunteers. This roadmap is very easy to parallelize, as it is comprised of many orthogonal features, so the faster we can power through it, the faster we can all benefit. We want these for our own pipelines too! 😆

If you'd like to get involved, hop in Discord or leave a comment on any of the issues linked above so we can coordinate. We're more than happy to help figure things out or pick up any work that you don't feel comfortable doing (e.g. UI, unfamiliar parts, etc.).

Thanks to everyone who has contributed so far, whether in code or in the community, and thanks to everyone for their patience while we figure out how to support such common functionality the "Concoursey way!" 🙏

Installation

Concourse is distributed as a single concourse binary, available on the Releases page.

If you want to just kick the tires, jump ahead to the Quick Start.

In addition to the concourse binary, there are a few other supported formats. Consult their GitHub repos for more information:

• Docker image
• BOSH release
• Kubernetes Helm chart

Quick Start

$ wget https://concourse-ci.org/docker-compose.yml
$ docker-compose up
Creating docs_concourse-db_1 ... done
Creating docs_concourse_1    ... done

Concourse will be running at 127.0.0.1:8080. You can log in with the username/password as test/test.

⚠️ If you are using an M1 mac: M1 macs are incompatible with the containerd runtime. After downloading the docker-compose file, change CONCOURSE_WORKER_RUNTIME: "containerd" to CONCOURSE_WORKER_RUNTIME: "houdini". This feature is experimental

Next, install fly by downloading it from the web UI and target your local Concourse as the test user:

$ fly -t ci login -c http://127.0.0.1:8080 -u test -p test
logging in to team 'main'

target saved

Configuring a Pipeline

There is no GUI for configuring Concourse. Instead, pipelines are configured as declarative YAML files:

resources:
- name: booklit
  type: git
  source: {uri: "https://github.com/vito/booklit"}

jobs:
- name: unit
  plan:
  - get: booklit
    trigger: true
  - task: test
    file: booklit/ci/test.yml

Most operations are done via the accompanying fly CLI. If you've got Concourse installed, try saving the above example as booklit.yml, target your Concourse instance, and then run:

fly -t ci set-pipeline -p booklit -c booklit.yml

These pipeline files are self-contained, maximizing portability from one Concourse instance to the next.

Learn More

• The Official Site for documentation, reference material, and example pipelines (which no longer live in this repository).
• The Concourse Tutorial by Stark & Wayne is great for a guided introduction to all the core concepts.
• See Concourse in action with our production pipelines
• Hang around in the GitHub discussions or in Discord.
• See what we're working on on the project board.

Contributing

Our user base is basically everyone that develops software (and wants it to work).

It's a lot of work, and we need your help! If you're interested, check out our contributing docs.

datadog-event-resource's Issues

What is the license for this repo?

Please and thank you!

Docker image latest tag is not being updated

Today I changed from my repository to concourse repository and I noticed that the latest tag for the docker image is not being updated with the latest merges, is this expected? In the readme it does not mention nothing regarding this so I would expect the latest tag to be updated at every merge?

Unable to `out` or `put` metrics events

We want to use the datadog resource to be able to do a put or an out on a metrics event. The current datadog resource does not allow for this functionality. It seems like the concourse team is also just curling against the endpoint manually: https://github.com/concourse/oxygen-mask/blob/c81da07e55ea062380d6717e4301da1ec6cc5599/monitor.yml#L92

It would be extremely useful to have the functionality for this implemented as well.

Certificate error after switch to alpine image

After the change to alpine based image I am not able to post events to datadog:

panic: Post https://app.datadoghq.com/api/v1/events?api_key=xxx&application_key=xxx: x509: failed to load system roots and no roots provided

goroutine 1 [running]:
main.main()
	/go/src/github.com/concourse/datadog-event-resource/cmd/out/main.go:42 +0x61d

I hijacked the container and found /etc/ssl was missing, so I:
apk add ca-certificates

Then I was able to curl https://app.datadoghq.com

Expose Concourse metatdata to tags

Is your feature request related to a problem? Please describe.

It would be great if we can expose the Concourse metadata e.g. $BUILD_ID, $BUILD_NAME, etc to tags

Describe the solution you'd like

When building the Concourse pipeline yml file, any variables under the tag section will not interpolate

- put: datadog-event
  params:
    title: Did you hear the news today?
    text: Oh boy!
    priority: normal
    tags:
    - environment:test
    - build-id:$BUILD_ID
    alert_type: info

Describe alternatives you've considered

No response

Additional context

No response

Allow connecting to datadog europe api

Hi, we use datadog europe datacenter and I noticed that this resource does not allow to connect to it.

In the go datadog api package, that this package uses, I noticed it is possible to change the datadog host by using the DATADOG_HOST env var, see here: https://github.com/zorkian/go-datadog-api/blob/9e2c892691035a7685a16d5839910cfc54e42fb6/client.go#L55.

I tried to set it on params on the get and put but didn't work.

Is there a way to do it? If it requires code implementation I could do it if you provide some guidelines.

Thanks in advance.

Feature Request: support build metadata

When emitting a datadog event, it would be helpful to include a link to the specific build that caused the event.

It would thus be useful to support text-substitution of the build metadata variables (https://concourse-ci.org/implementing-resources.html#resource-metadata).

Alternatively, it would be useful to support text-substitution of the build URL directly.