cobyism / dciy Goto Github PK

Once a build finishes, this project should integrate with GitHub’s Status API by sending the build status (success, failure etc.) to GitHub for the given commit to be marked appropriately, along with a URL attached to the commit status so that you can be taken to DCIY to view the build output.

Given DCIY is run locally, and the link in the commit status from GitHub is coming from the internet, this may get difficult if there are multiple people collaborating on the same project, but each running DCIY themselves locally. This project is probably more designed for just being run by a single person though, so that issue might not be something I want to do anything about (unless there’s an obvious solution). That shouldn’t prevent integration with the status API by any means though.

If Strider can do builds on Heroku by cloning the repo to the ephemeral filesystem of the running dyno, then DCIY could probably be used on Heroku using a similar technique. I’d love it if this project where able to do this 😄

It would be nice to do be able to do dev builds from a working git to a local vagrant.

I’d love to create a simple one-pager site for DCIY using GitHub Pages to better convey what this thing is, who it’s for, how to get it up and running, and how to contribute.

When a build causes an internal DCIY exception, right now it fills the build output with a giant stack trace:

A better idea is to catch any thrown exceptions, store them on the Build object, and show them below. We can have "friendly text" for the expected ones (like BranchNotFoundError) and show the full trace in an initially-collapsed <div>.

Relocated from a discussion on #33.

Right now the only way to trigger a build is via DCIY’s web interface itself, but ideally builds should be triggered each time a push is made to GitHub for a project.

Things to consider:

• Do we automatically set up new projects if a payload is received?
• Making DCIY accessible from the internet may be hard for some people to work out how to do—documentation here will be important.
• If, people will be putting DCIY up on the internet publicly how to protect DCIY from malicious POSTs, or from people trying to look at it who shouldn’t have access.
• How DCIY knows about its own location on the public internet, and how to structure URLs in commit statuses (/cc #2) so that both the person running DCIY and other people collaborating on the project can all access build output etc. easily (going to a public URL for the project running locally may not resolve DNS and so forth)
• Can we automate the setting up of webhooks? Do we need API access to do this? do we just show people a webhook URL to copy/paste from DCIY with a token in it and explain where people should go in GitHub to set up the hook?

As per discussion on #34. Since it's basically an engine for running arbitrary code off the Internet, DCIY is, er, kind of a giant security hole. To mitigate this, let's run builds in Docker containers.

Some considerations:

• Docker doesn't run natively on Macs (or Windows), but in Vagrant container. It'll make setup a little more involved. Should script/bootstrap take care of platform detection and installation and everything?
• Users will need to configure docker to have a proper setup, as opposed to just using the native environment. Although we could do the fetch and the GitHub API communication with non-Docker commands, and just execute prepare and cibuild commands in Docker-land.

If DCIY is designed to ever be put up publicly on the internet, then a cool feature to add would be to have build status badge images available for each project to indiciate passing/failing status of the primary branch of any project. This may require protecting private repositories or something, so that it’s not possible to discover anything about a project or it’s existence from a badly set up public DCIY instance.

There is a problem building DCIY master branch when running DCIY on master. Output from rspec is not shown, and the build completes as if bundle exec rake spec returned successfully without doing anything. Here is the output from DCIY:

This does not seem to happen when cloning and building a fresh copy of DCIY from terminal, although that gets a lot of rspec errors due to the security token not being set.

Sorry I don't have time to look into this further this morning. My next steps would be testing against another code base to see if that builds properly, and then doing a bisect to see where this was introduced if it's not just a problem with our own build configuration.

I am wondering whether there's some strange cache issue with regard to build output being updated as they run. I kept hitting refresh, would get more output in the window, then it would disappear:

Screen recording

Once the build was done, I could successfully refresh and see all the output without it disappearing.

Visiting builds_path after destroying a project which had generated builds results in a NoMethodError. In the Builds#index view, it tries to access build.project.repo which raises an error when trying to call repo on the undefined build.project.

I think it makes sense to cascade the destroy action to builds attached to the project. We can accomplish with a small migration - needs regression test.

I’ve just been build this up from scaffolds, and this project really needs a test suite of it’s own. Bonus points if DCIY can be set up to build itself as a special-case part of the UI or something to make hacking on it easier (i.e. while the dev server is running). Recursion is awesome.

It would be nice if it were possible to configure DCIY to automatically deploy certain branches of a project to heroku (or any other git remote, for that matter) if a commit at the tip of that branch passes CI. Again, following the philosophy of "if you can run CI locally, then DCIY can do it for you", then this makes sense because "if you can deploy to heroku locally, then DCIY should be able to do it for you".

Maybe this just needs to be a new model (something like BranchDeployer) that belongs to each Project, and knows about:

• The branch name to watch for successful builds
• The remote name and URL to push to (note that the CI workspace won’t have the same remotes set up after clone as your dev directory of the same project).
• The destination branch name on the remote repository to push to.

For example:

• Set up the master branch so that successful builds are pushed to the master branch of the production git remote (which should have the URL of [email protected]:<project-production>.git)—i.e. git push production master:master
• Set up the dev branch so that successful builds are pushed to the master branch of the staging git remote (which should have the URL of [email protected]:<project-staging>.git)—i.e. git push staging dev:master

Add in UI for branch-deploying successfully built topic branches to remote destinations, for example if you want to test out the awesome-new-feature branch on production, then you should be able to do that as a once-off (as long as the branch HEAD is green) and then redeploy master back when you’re finished.

Then again, maybe the whole deployment problem needs to be a whole other application—maybe it’s best to just keep DCIY focused on the job of listening for new commits (/cc #5), building them, and telling GitHub’s status API about the result (/cc #2). Another app could listen for the status API success notification, detect the branch/deployment rules, and have a separate set of UI focused on just that.

Visiting the new_project_path and submitting the form with an empty repository URL succeeds in creating a new project. More so, builds can be queued from this blank project, but thankfully they fail within 30 seconds when a NoMethodError is thrown.

Build output:

[DCIY] Aight, let's do this!
[DCIY] Cloning repository...
[DCIY] Looking up SHA for branch 'master'
[DCIY] SHA to build is fatal:
[DCIY] Checking out project at fatal:...
NoMethodError: undefined method `find_head' for #<Runner:0x007f887ab290d0>

/Users/jwilm/code/dciy/app/models/runner.rb:173:in `sha_for_branch'
/Users/jwilm/code/dciy/app/models/runner.rb:75:in `do_checkout'
/Users/jwilm/code/dciy/app/models/runner.rb:24:in `run_run_run'
/Users/jwilm/code/dciy/app/models/runner.rb:9:in `go_nuts_on'
/Users/jwilm/code/dciy/app/workers/build_worker.rb:13:in `perform'

...

I see two solutions with varying robustness/complexity.

1. We can validate the repository URL is technically valid with a regex matcher. This does not guarantee the repository exists.
2. Query the GitHub API to check that the repository exists. I guess the benefit here is that a user cannot put a typo in their URL and wonder why their builds are failing.

This is all currrently just a Rails scaffold UI, so this really needs a proper UI designed for it. There’s really only a handful of screens needed:

• A list of all builds, and the root URL.
• A list of builds for a specific project.
• The create/configure project form.
• The build triggering form.
• The individual build result page.

Switch language runtime (for example, to Ruby 1.9.3 or PHP 5.4)

• should respect .ruby-version via rbenv or RVM
• node npm etc.

Right now, the project only runs CI by executing script/cibuild inside each project, but it’s obvious that many projects will want different commands to be run instead to execute the bootstrap/test run for their specific project.

This probably requires being able to configure this at the project level, so there’s a couple of ways this could be achieved:

• it might just need a field to be added to each Project where people can specify the command to be run. The downside of this is it lives in the database, rather than in the project itself.
• maybe this project simply mandates a specific file convention (similar to travis.yml) where information about what should be run is stored.

I’m going to say that the later option (mandating a dciy.toml or something) might be the most flexible, simple solution.