concourse / dex

A fork of coreos/dex with changes necessary for Concourse. **See `maintenance` branch for details.**

Concourse: the continuous thing-doer.

Concourse is an automation system written in Go. It is most commonly used for CI/CD, and is built to scale to any kind of automation pipeline, from simple to complex.

Concourse is very opinionated about a few things: idempotency, immutability, declarative config, stateless workers, and reproducible builds.

The road to Concourse v10

Concourse v10 is the code name for a set of features which, when used in combination, will have a massive impact on Concourse's capabilities as a generic continuous thing-doer. These features, and how they interact, are described in detail in the Core roadmap: towards v10 and Re-inventing resource types blog posts. (These posts are slightly out of date, but they get the idea across.)

Notably, v10 will make Concourse not suck for multi-branch and/or pull-request driven workflows - examples of spatial change, where the set of things to automate grows and shrinks over time.

Because v10 is really an alias for a ton of separate features, there's a lot to keep track of - here's an overview:

Feature	RFC	Status
set_pipeline step	✔ #31	✔ v5.8.0 (experimental)
Var sources for creds	✔ #39	✔ v5.8.0 (experimental), TODO: #5813
Archiving pipelines	✔ #33	✔ v6.5.0
Instanced pipelines	✔ #34	✔ v7.0.0 (experimental)
Static across step	🚧 #29	✔ v6.5.0 (experimental)
Dynamic across step	🚧 #29	✔ v7.4.0 (experimental, not released yet)
Projects	🚧 #32	🙏 RFC needs feedback!
load_var step	✔ #27	✔ v6.0.0 (experimental)
get_var step	✔ #27	🚧 #5815 in progress!
Prototypes	✔ #37	⚠ Pending first use of protocol (any of the below)
run step	🚧 #37	⚠ Pending its own RFC, but feel free to experiment
Resource prototypes	✔ #38	🙏 #5870 looking for volunteers!
Var source prototypes		🚧 #6275 planned, may lead to RFC
Notifier prototypes	🚧 #28	⚠ RFC not ready

The Concourse team at VMware will be working on these features, however in the interest of growing a healthy community of contributors we would really appreciate any volunteers. This roadmap is very easy to parallelize, as it is comprised of many orthogonal features, so the faster we can power through it, the faster we can all benefit. We want these for our own pipelines too! 😆

If you'd like to get involved, hop in Discord or leave a comment on any of the issues linked above so we can coordinate. We're more than happy to help figure things out or pick up any work that you don't feel comfortable doing (e.g. UI, unfamiliar parts, etc.).

Thanks to everyone who has contributed so far, whether in code or in the community, and thanks to everyone for their patience while we figure out how to support such common functionality the "Concoursey way!" 🙏

Installation

Concourse is distributed as a single concourse binary, available on the Releases page.

If you want to just kick the tires, jump ahead to the Quick Start.

In addition to the concourse binary, there are a few other supported formats. Consult their GitHub repos for more information:

• Docker image
• BOSH release
• Kubernetes Helm chart

Quick Start

$ wget https://concourse-ci.org/docker-compose.yml
$ docker-compose up
Creating docs_concourse-db_1 ... done
Creating docs_concourse_1    ... done

Concourse will be running at 127.0.0.1:8080. You can log in with the username/password as test/test.

⚠️ If you are using an M1 mac: M1 macs are incompatible with the containerd runtime. After downloading the docker-compose file, change CONCOURSE_WORKER_RUNTIME: "containerd" to CONCOURSE_WORKER_RUNTIME: "houdini". This feature is experimental

Next, install fly by downloading it from the web UI and target your local Concourse as the test user:

$ fly -t ci login -c http://127.0.0.1:8080 -u test -p test
logging in to team 'main'

target saved

Configuring a Pipeline

There is no GUI for configuring Concourse. Instead, pipelines are configured as declarative YAML files:

resources:
- name: booklit
  type: git
  source: {uri: "https://github.com/vito/booklit"}

jobs:
- name: unit
  plan:
  - get: booklit
    trigger: true
  - task: test
    file: booklit/ci/test.yml

Most operations are done via the accompanying fly CLI. If you've got Concourse installed, try saving the above example as booklit.yml, target your Concourse instance, and then run:

fly -t ci set-pipeline -p booklit -c booklit.yml

These pipeline files are self-contained, maximizing portability from one Concourse instance to the next.

Learn More

• The Official Site for documentation, reference material, and example pipelines (which no longer live in this repository).
• The Concourse Tutorial by Stark & Wayne is great for a guided introduction to all the core concepts.
• See Concourse in action with our production pipelines
• Hang around in the GitHub discussions or in Discord.
• See what we're working on on the project board.

Contributing

Our user base is basically everyone that develops software (and wants it to work).

It's a lot of work, and we need your help! If you're interested, check out our contributing docs.

dex's Issues

gitlab connector assigns name as username

In
https://github.com/concourse/dex/blob/master/connector/gitlab/gitlab.go:187

one can find

	ident.Name = name
	ident.Username = name
        ident.Email = user.Email

probably should be

ident.Username  = user.Username

which seems to be incorrect assingment

Tracking deltas from upstream coreos/dex

It looks like we'll be sitting on this fork for a while, so I'm gonna try and keep track of all our PR-able changes here before things get too hairy.

I'm also gonna replace all the import paths from coreos/dex to concourse/dex and make this a proper fork until we can get those PRs merged. We're relying on submodules in the concourse/concourse repo for this import path repointing, but that won't work with Go 1.11's modules, which I'd like to start using.

Here's the plan:

1. Create a new branch called upstream which will point to the coreos/dex upstream master that all local branches are rebased on.
2. Create new branches for reach of our PR-able changes, based on upstream. This way they can be submitted individually rather than all being merged at once.
3. Reset master to upstream master.
4. Delete all branches but master and the local PR branches (we inherited a lot of'em from upstream and it's confusing to keep them).
5. Maintain each local branch for as long as needed to get merged upstream (this may involve rebasing on upstream and force-pushing).
6. Maintain master as one big merge of all local branches and on top of upstream (this will definitely involve force-pushing, as local branches are constantly rebased).
7. Any time we re-push master, put a commit on top that re-points all coreos/dex import paths to concourse/dex.
8. At the same time, tag HEAD with something so that the ref doesn't go away (thereby breaking old submodule pointers in concourse/concourse).

The downside of this is that master will be force-pushed all the time. The upside is that go get should "just work", all import paths will point at our fork, and all our changes will be PR-able.

Using self-signed certificate in OIDC fails x509 to get UserInfo

Preflight Checklist

• I agree to follow the Code of Conduct that this project adheres to.
• I have searched the issue tracker for an issue that matches the one I want to file, without success.
• I am not looking for support or already pursued the available support channels without success.

Version

3.1.0

Storage Type

In-memory

Installation Type

Other (specify below)

Expected Behavior

This issue is related to dex used internally in Concourse.
Expected behavior is that you can login into Concourse with OIDC and self-signed certificate with skipSslValidation set to true.

Actual Behavior

Issue occurs in Concourse version 7.8.0 and higher (where new flag was introduced: CONCOURSE_OIDC_DISABLE_GET_USER_INFO).
This flag by default is set to false (DisableGetUserInfo is set to false) which means it will try to get UserInfo. But it fails when self-signed cert is used, as in the image below:

Steps To Reproduce

1. Configure concourse with OIDC: skipSslValidation set to true and disableGetUserInfo set to false.
2. Try to login
3. You get an error about self-signed x509 certificate

Additional Information

This is probably caused by missed one line where http.Request context is used instead of the context with local httpClient:
https://github.com/concourse/dex/blob/v0.8.0/connector/oidc/oidc.go#L295

I am not sure if this should be reported in concourse repository as well.

Configuration

No response

Logs

No response

Support for Google connector?

Concourse's Dex is way behind the upstream (328 commits behind dexidp:master). Is there any plans to put it up to date and have the ability to connect with the Google connector?

[CF Connector] Use the `authorization_endpoint` instead of `token_endpoint` from /v2/info

Apparently they do different things.

Problems with GitLab connector

The default URL for the GitLab connector https://www.gitlab.com does not seem to work any more. I opened a small PR on coreos/dex (dexidp#1279) but the repo does not seem to evolve right now. Maybe it's interesting for this fork as well.

Possibility of pass the access token on oAuth to the userinfo url?

Note: I'm very unfamiliar with Go and I don't know how much effort that would be, but I guess it's just a small change to

dex/connector/oauth/oauth.go

Line 156 in 2202f4e

userInfoResp, err := client.Get(c.userInfoURL)

I want to use oAuth with Phabricator as server. Acquiring the code and the token seems to work fine. At least I dont get any errors or warnings ;)

However, for retrieving the user info from Phabricator, /api/user.whoami?access_token=<token> is required to be called. Is it possible to implement this in the oAuth connector? Currently, the access token is not passed to the server on retreiving user info.