fabric8-launcher / launchpad-missioncontrol Goto Github PK

Fix noisy logging:

if(log.isLoggable(Level.FINEST)) {
         log.finest("OAuth2 response headers:");
         response.getHeaders().forEach((key, values) -> log.info(key + ": " + values));
      }

Utilize the fabric8-arquillian to facilitate integration testing

• Configure project to support Arquillian
• Migrate existing IT tests to Arquillian framework

https://developer.github.com/v3/oauth/#web-application-flow

The entry point to a user looking to Get Started with an Example Application looks like:

• Developer is investigating technologies: http://developers.redhat.com/products/eap/get-started/
• http://developers.redhat.com - Quickstart button that triggers setup
• CI/CD Webapp forks the Quickstart authoritative repo to the user’s namespace on GitHub

With #4 we have the beginnings of a GitHub Service which can fork the repository into the user's namespace; this assumes we know the user's loginID (username) and OAuth token. The link above describing the GitHub API's "Web Application Flow" dictates how we may get the OAuth token; along the way it'll prompt the user to grant us access and show the permissions we're requesting.

So what we need is to start to build a webapp to implement the OAuth Web Flow, here:

https://github.com/redhat-developer-tooling/katapult/tree/master/web/src/main/java/org/rhd/katapult/web

There is a PoC that @gastaldi did with some relevant code here:

https://github.com/redhat-developer-tooling/katapult/blob/master/services/github-service/src/main/java/org/rhd/katapult/github/GithubResource.java

...though we'll refactor/retire this bit as necessary.

Ideally this task will also encompass Arquillian tests with Drone/Graphene (http://arquillian.org/guides/functional_testing_using_graphene/) to simulate the user interaction with the browser as they click through the OAuth permissions.

Probably into core-api/test where it can be imported as a test resource and used elsewhere.

The linked page on openshift origin offers a few options to get started, be more explicit about which one.

Add to the README in the project root the scopes we need to enable when creating the GitHub personal access token. Screenshot below shows the scopes to enable; this task is just about documenting it.

Using a GitHub account with 2-factor auth disabled (standard username/pass login). Executed by: mvn clean install -Pit (Arquillian WildFly 10.0.0.Final Managed Adaptor)

Test fails with timeout:

should_fork_jboss_eap_quickstarts(org.rhd.katapult.test.services.GithubResourceIT)  Time elapsed: 98.85 sec  <<< ERROR!
com.gargoylesoftware.htmlunit.ScriptException: Exception invoking setHref
    at org.rhd.katapult.test.services.GithubResourceIT.should_fork_jboss_eap_quickstarts(GithubResourceIT.java:77)
Caused by: java.lang.RuntimeException: Exception invoking setHref
    at org.rhd.katapult.test.services.GithubResourceIT.should_fork_jboss_eap_quickstarts(GithubResourceIT.java:77)
Caused by: java.lang.RuntimeException: org.apache.http.conn.ConnectTimeoutException: Connect to 192.168.1.107:8080 [/192.168.1.107] failed: connect timed out
    at org.rhd.katapult.test.services.GithubResourceIT.should_fork_jboss_eap_quickstarts(GithubResourceIT.java:77)
Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to 192.168.1.107:8080 [/192.168.1.107] failed: connect timed out
    at org.rhd.katapult.test.services.GithubResourceIT.should_fork_jboss_eap_quickstarts(GithubResourceIT.java:77)
Caused by: java.net.SocketTimeoutException: connect timed out
    at org.rhd.katapult.test.services.GithubResourceIT.should_fork_jboss_eap_quickstarts(GithubResourceIT.java:77)

Server hangs here:

2016-04-07 17:37:50,511 INFO  [org.wildfly.extension.undertow] (ServerService Th
read Pool -- 24) WFLYUT0021: Registered web context: /katapult
2016-04-07 17:37:50,568 INFO  [org.jboss.as.server] (management-handler-thread -
 3) WFLYSRV0010: Deployed "application-ear.ear" (runtime-name : "application-ear
.ear")
2016-04-07 17:37:51,659 INFO  [org.rhd.katapult.github.GithubResource] (default 
task-1) Request to fork repository: jboss-developer/jboss-eap-quickstarts
2016-04-07 17:37:52,287 INFO  [org.rhd.katapult.github.GithubResource] (default 
task-1) OAuth2 response: Found

Pending approval from GitHub support if we can make these public.

By doing so, we'd be able to enable anyone forking the repository to run the tests; else they'd need to have access to an OAuth Client ID and Secret of their own with callback URL to http://localhost:8080/katapult/api/github/callback for local testing

Currently set to "Github forker" or "Forge", should be something more like:

"Kontinuity Catapult by Red Hat"

Likely backed by the Fabric8 openshift-client

For this task, just stub out an API for ourselves to show the ops we support, implement a a couple of calls into OpenShift, and write tests proving it all works. In subsequent tasks we can get more fine-grained due to features needed but this one is primarily about setting up a working service module for us to use.

"Kontinuity" is the public effort for Red Hat's CI/CD efforts aimed at developers.

This project is the launcher for that initiative, so we'll call it "Catapult", under the umbrella of Kontinuity.

Rename the project (and the GAVs). This will require everyone to update their remotes to point to the rename.

Let's get rid of a lot of the custom ServiceLoader code and rely upon the container to inject an instance.

Tests should be done in stages: a base that has an extension to provide the GitHubServiceFactory as a POJO, and another test to run against the container getting the GitHubServiceFactory injected as an EJB or CDI managed component

Miscellaneous code/doc fixes to address Sonar warnings

Currently:

GitHubRepository create(String repositoryName, String description, String homepage,
        boolean has_issues, boolean has_wiki, boolean has_downloads) throws IOException, IllegalArgumentException

...the impl has:

@Override
    public GitHubRepository createRepository(String repositoryName,
                                   String description,
                                   String homepage,
                                   boolean hasIssues,
                                   boolean hasWiki,
                                   boolean hasDownloads) throws IOException, IllegalArgumentException

Let's:

1. Drop the IOException from the SPI and on the impl side wrap it in a RuntimeException
2. Ensure there's test coverage in GitHubServiceIT
3. Drop the "has" issues, wiki, and downloads from what we provide (we'll set these flags to false on the impl side). The user can always add these if they want from the GitHub Web UI, but we won't ever be prompting them for this info if we make new repos on their behalf.
4. Drop homepage

Basically let's get the SPI to read:

/**
     * Creates a repository with the given information.  Issues, Wiki, and Downloads will be set to disabled.
     *
     * @param repositoryName - the name of the repository
     * @param description - the repository description
     * @return the created {@link GitHubRepository}
     * @throws IllegalArgumentException
     */
GitHubRepository createRepository(String repositoryName, String description) throws IllegalArgumentException

We can't share our OAuth app's Client ID and Secret, so document for a developer of Catapult how they may. Replaces #38.

Priority should be (lower numbers means higher priority):

1. System Property {@link org.kontinuity.catapult.service.openshift.api.OpenShiftEnvVarSysPropNames#OPENSHIFT_URL}
2. Environment Variable {@link org.kontinuity.catapult.service.openshift.api.OpenShiftEnvVarSysPropNames#OPENSHIFT_URL}
3. Current instance of OpenShift we are running in
4. {@link OpenShiftApiUrl#DEFAULT_OPENSHIFT_URL}

Also refactor the CatapultIT test case to use injection inside WildFly

Create an extension of the OpenShiftService API for testing which adds in support for deletion of OpenShift projects.

Will be used by tests only for the time being for cleanup operations after tests run.

Also call this in the OpenShiftServiceIT test to clean up all created projects.

When we fork a repository for a user, we need to also add a webhook on that repository to hit the OpenShift project to trigger the build pipeline on new pushes.

Core API and entry point to the Kontinuity Catapult. Defines high-level capabilities intended to be called by outside clients; designed to be additionally exposed as a web endpoint.

This is where we need to determine the inputs for what we need to:

• Fork a GitHub repo into a user's namespace
• Create an OpenShift project for the user
• Set up the project with the right template (pipeline build config), repo reference
• Register with GitHub a webhook to point to the OpenShift project

Also this will define the output for what we return to callers

• Location of the new Project in OpenShift
• ...etc

This task simply to set up the API and a first-pass impl. Further work and iterations done here may be completed in new, finer-grained issues. For now let's just give ourselves a place to work so we can hook things together. The web pieces will expose this API as a RESTful (or at least HTTP, to be determined) service.

We're currently using in the Fabric8OpenShiftClientServiceImpl:

withTrustCerts(true)

This can't stand in production (though is fine for local testing); we must have the certificate in the trust store for stage testing and production deployment. Resolve this before going live.

We don't need the HTTP Transport URL or the short name

Let's get rid of a lot of the custom ServiceLoader code and rely upon the container to inject an instance.

Tests should be done in stages: a base that has an extension to provide the OpenShiftServiceFactory as a POJO, and another test to run against the container getting the OpenShiftServiceFactory injected as an EJB or CDI managed component

[ERROR] Failed to execute goal on project catapult-tests: Could not resolve dependencies for project org.kontinuity.catapult:catapult-tests:jar:1.0.0-SNAPSHOT: Could not find artifact org.kontinuity.catapult:catapult-core-api:jar:tests:1.0.0-SNAPSHOT -> [Help 1]

These no longer exist after the module refactoring done in ba7a87f

Centralize how we might get these in a test environment.

From GithubResourceIT:

@BeforeClass
    public static void initGithubCredentials() throws IOException {
        GITHUB_USERNAME = System.getenv(ENVVAR_NAME_GITHUB_USERNAME);
        Assume.assumeNotNull("Could not find env var " + ENVVAR_NAME_GITHUB_USERNAME, GITHUB_USERNAME);
        GITHUB_PERSONAL_ACCESS_TOKEN = System.getenv(ENVVAR_NAME_GITHUB_TOKEN);
        Assume.assumeNotNull("Could not find env var " + ENVVAR_NAME_GITHUB_TOKEN, GITHUB_PERSONAL_ACCESS_TOKEN);
    }

We'll want to prioritize sysprops over env vars.

core-api should depend on APIs only.

At a minimum, instructions to build and run the tests

Currently we have in the tests a hardcoded:

private static final String URL_OPENSHIFT = "https://localhost:8443";

We should be employing a scheme where we:

1. Default like we currently have
2. Determine if we're running in OpenShift and if we are, use that
3. Look to an env var
4. Look to a sysprop

Depends on #49 so we can inject the service into the RESTful component

Depends on actually creating a GitHubServiceSpi, reflected by #61

This cannot stay in the API:

/**
     * Deletes all webhooks in a specific GitHub repository
     * 
     * @param repository - the value object that represents the GitHub repository
     * @throws IOException
     * @throws IllegalArgumentException If the parameter is unspecified
     */
    void deleteWebhooks(final GitHubRepository repository) throws IllegalArgumentException;

Needs also to reflect not deletion of all webhooks, but of a single webhook; in tests we only want to delete the webhook we created, not all.

We should create a new repo, put a webhook in it, and delete the entire repo; this will:

1. Not bind this test to a dependency upon a source repository ( a remote resource)
2. Ensure we don't accidentally remove webhooks that we didn't create (as we are currently doing by removing all webhooks for a repo)

. Means we need to add SPI support to delete a repository.

Depends on #61

Will have pipeline instructions to power the CI job.

@rawlingsj has a starting point for us: 18eb837

core-api should depend upon APIs of other modules only

For @rimolive to validate.

I'm pulling some code to this effect out of the work done in #40 to isolate that task to the GitHubService. Can we outline the requirement here for why the OpenShiftService needs:

/**
 * Return the Github WebHook from an application.
 *
 * @param namespace
 * @param applicationName
 * @return
 */
String getGithubWebhook(String namespace, String applicationName);

Our CI pipeline will be staged, so we'll want to be running first a Maven build to create our project, then get it deployed into OpenShift, and then another Maven build to run the integration tests.

Create the profile that supports this and move integration testing into it.

*_/Test.java > Unit tests
*/_IT.java > Integration tests

http://jenkins.distortion.origin.fabric8.io/job/catapult/3/testReport/

To support:

• Deleting a repository
• Deleting a WebHook
• Creating a repository

...to be used in testing

Always seems to be using "phantomjs" regardless of if the "chrome" or "firefox" profile is specified.

Should point to the URL for this project in the console, likely:

OpenShiftUrl.get() + "/console/project/{projectName}/overview"

...but is there a way to get this from the OpenShift API or are we best doing the concatenation on our own?

Ultimately this would be used to redirect the user to the project overview page when we're done.

Currently we log in and make projects as a user; instead we really need to, as an admin, make projects for users and grant them access.

dependency-management is fine; move dependency declarations to modules where they are needed. No unused dependencies leaking.

We've got things working but a bit more hardcoded than we'd like, so let's split them out.