saucelabs / sauce-java Goto Github PK

See http://support.saucelabs.com/tickets/4050 for details

Hi,
i've ran the maven sauce-connect-example and found out sc doesn't get killed via:

                    <execution>
                        <id>stop-sauceconnect</id>
                        <phase>post-integration-test</phase>
                        <goals>
                            <goal>stop-sauceconnect</goal>
                        </goals>
                    </execution>

Also tried with latest version (2.1.4)
Can anyone verify & also advise on this behavior?

Thanks

We're using the sauce-connect-plugin to run Selenium tests in our Maven build. Sauce Connect seems to be pretty chatty, though, and I haven't found a clean way to control the output from Maven. It seems there are at least two types of output to control...first is the java.util.logging output coming from SauceConnectTwoManager's StreamGobbler, which I've managed to control if I pass a system property on the Maven command line (-Djava.util.logging.config.file=...) pointing to a logging config file. This reduces some of the output, but requires us to pass a lengthy command line argument whenever we build, since Maven makes it inordinately difficult to set a system property globally from within the build configuration.

Second is some much more verbose output that looks like wire-level request logging of GETs and POSTs to SauceLabs for individual browser interactions. I'm not sure yet where this is being output, but it's now the overwhelming majority of output produced by our build.

Am I missing some way of configuring the logging? Many Maven plugins have a way of passing in system properties in a configuration element, which the sauce-connect-plugin seems to be lacking. That would help control the SauceConnectTwoManager output. It seems another mechanism is also needed, presumably to control output from the SauceConnect subprocess?

Thanks.

It can actually be a little error prone to stop a sauce session because we can get back a null driver for multiple reasons. Let's wrap this command into a nice method for our clients so that it's not error prone and easier to use.

            if(_driver != null)
                ((IJavaScriptExecutor)_driver).ExecuteScript("sauce:job-result=" + (passed ? "passed" : "failed"));
            _driver?.Quit();

NPE in SauceConnectFourManager when running with sauce connect while running with true useLatestSauceConnect flag, it tries to find sc-4.4.12 but returned resource is null.

java.lang.NullPointerException
2018-03-03 14:40:13,436 [pool-2-thread-1] INFO org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2314)
2018-03-03 14:40:13,436 [pool-2-thread-1] INFO org.apache.commons.io.IOUtils.copy(IOUtils.java:2270)
2018-03-03 14:40:13,437 [pool-2-thread-1] INFO org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2291)
2018-03-03 14:40:13,437 [pool-2-thread-1] INFO org.apache.commons.io.IOUtils.copy(IOUtils.java:2246)
2018-03-03 14:40:13,437 [pool-2-thread-1] INFO com.saucelabs.ci.sauceconnect.SauceConnectFourManager.extractFile(SauceConnectFourManager.java:379)
2018-03-03 14:40:13,437 [pool-2-thread-1] INFO com.saucelabs.ci.sauceconnect.SauceConnectFourManager.extractZipFile(SauceConnectFourManager.java:302)

It's a little annoying to be doing something like this as a user:

        WebDriver driver = new RemoteWebDriver(new URL(SAUCE_REMOTE_URL), caps);
        SessionId sessionId = ((RemoteWebDriver) driver).getSessionId();

It would be way easier if our client could automatically set this property and then we could use it like this:

        SauceHelper sauceHelper = new SauceHelper(driver);
sauceHelper.getSessionId();

The following was reported to Sauce Labs support, reiterating here for consideration.

the TestNG listener @listeners({SauceOnDemandTestListener.class}) only
works when the tests are methods oriented. When the tests are classes
oriented it doesn’t work, because the result of the last method that
matters. Example: If You have 1 class with 5 methods and the first 4 FAIL
and the last one PASS, so the final result is PASS for whole the class test.

Workaround: I am not using @Listener. I am using sauceRest. You can keep
two @Listener version, one for methods and another for classes oriented.

@afterclass(alwaysRun = true)

public void tearDown() {

SauceREST sauceREST = new SauceREST(USUARIO_SAUCELAB,
ACCESSKEY_SAUCELAB);

if(isTestSuccesful) {

sauceREST.jobPassed(getSessionId());

} else {

sauceREST.jobFailed(getSessionId());

}

driver.quit();

}

@AfterMethod

public void avaliaResultadoTeste(ITestResult result) {

if(!result.isSuccess()) {

isTestSuccesful = Boolean.FALSE;

}

}

@rossrowe

I am using sauce-connect-plugin in my project

                                           <groupId>com.saucelabs.maven.plugin</groupId>
                    <artifactId>sauce-connect-plugin</artifactId>
                    <version>2.1.13</version>

But, I have had problems with sauce connect, as the session connects and disconnects several times. I contacted the support of SauceLabs and he asked me to change the sauce connect 4.3 to 4.3.9.
Could You update the plugin?

Thanks in advance,

Ruben Rose

========== saucelabs email ============================
Good Afternoon Ruben,
I noticed we are using Sauce Connect 4.3. Sauce Connect has gone through a lot of versions since then and I’d like to make sure this isn’t a problem we have solved in those iterations.
Could you please grab the latest version 4.3.9 from https://docs.saucelabs.com/reference/sauce-connect/ and try this again?
Colin

Hi guys,
not sure if I just don't see it or if it's really not there.

I miss an option to set a proxy in the sauce-connect-plugin config.
I only have the option to run my project with -p http://my.proxy:8080 but this is not an option in some cases for me. What I would really appreciate would be something like:

<plugin>
    <groupId>com.saucelabs.maven.plugin</groupId>
    <artifactId>sauce-connect-plugin</artifactId>
    <version>2.1.13</version>
        <configuration>
            <sauceUsername>user</sauceUsername>
            <sauceAccessKey>accesskey</sauceAccessKey>
            <http_proxy>http://my.proxy:8080</http_proxy>
            <https_proxy>http://my.proxy:8080</https_proxy>
        </configuration>
...
</plugin>

What do you say?

The pom.xml in the TestNG example is pointing to a SNAPSHOT. Using 2.1.20 will find the correct dependency and compile.

Check for session being null before trying to update status.

v2.1.11 doesn't exist on the Central Repository. Please make it available!

I would like to avoid adding external repositories to my pom.xml as Maven doesn't handle them particularly well (it consults all repositories for all artefacts) and they can sometimes contain bad copies of artefacts. This leads to a less reliable and slower build.

Is there any chance you could add the Sauce Connect Maven plugin to Maven Central?

When running tests remotely with SauceLabs with "SELENIUM_IS_LOCAL=false". It actually happens with setting it to "True" too, but that's documented in a separate issue already: #51

Below is the stack trace:

java.lang.NullPointerException at com.saucelabs.testng.SauceOnDemandTestListener.markJobStatus(SauceOnDemandTestListener.java:115) at com.saucelabs.testng.SauceOnDemandTestListener.onTestSuccess(SauceOnDemandTestListener.java:142) at org.testng.internal.TestListenerHelper.runTestListeners(TestListenerHelper.java:70) at org.testng.internal.TestInvoker.runTestResultListener(TestInvoker.java:210) at org.testng.internal.TestInvoker.invokeMethod(TestInvoker.java:638) at org.testng.internal.TestInvoker.invokeTestMethod(TestInvoker.java:172) at org.testng.internal.MethodRunner.runInSequence(MethodRunner.java:46) at org.testng.internal.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:804) at org.testng.internal.TestInvoker.invokeTestMethods(TestInvoker.java:145) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:128) at java.base/java.util.ArrayList.forEach(ArrayList.java:1540) at org.testng.TestRunner.privateRun(TestRunner.java:770) at org.testng.TestRunner.run(TestRunner.java:591) at org.testng.SuiteRunner.runTest(SuiteRunner.java:402) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:396) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:355) at org.testng.SuiteRunner.run(SuiteRunner.java:304) at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53) at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:96) at org.testng.TestNG.runSuitesSequentially(TestNG.java:1180) at org.testng.TestNG.runSuitesLocally(TestNG.java:1102) at org.testng.TestNG.runSuites(TestNG.java:1032) at org.testng.TestNG.run(TestNG.java:1000) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:135) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:193) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:94) at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:146) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)

The NPE seems to occur because of this line in the SauceOnDemandTestListener (because the sauceREST variable has not been initialized):

That "sauceREST" variable gets initialized here, under this condition based on this.isLocal:

this.isLocal gets set to "true" here (the default value is "false"). The problem is that passing any value in for SELENIUM_IS_LOCAL will cause this method to set isLocal to "true". The only way to get isLocal to be "false" is by passing in a null/empty string:

It seems the only way to run remotely and avoid this stack trace is to use "SELENIUM_IS_LOCAL=" or not pass the parameter at all (even though passing "true" will also produce the NPE).

I added a new class called SauceHelperTests in com.saucelabs.common. This class has some unit tests. The unit tests are not executing and code coverage isn't picking them up. Looks like we need to make some update to the CI build configuration.

I'm having an issue with the Sauce Connect Plugin when trying to start up a tunnel for my tests.

start-sauceconnect is able to reach out to SauceLabs and create a tunnel for me, but once it gets to a phase where it's started the SeleniumListener on port 4445, it's immediately shutting the tunnel back down, then starting my tests. When starting sc from the command line however, I am able to fully connect with the same command line settings when I do it manually.

Attached is the output log from SauceConnect
sc_localComputer.txt

Any ideas?

I would like to speed up test execution for our cod base and make all the tests run in parallel. Not sure how to do that since I'm a Java novice. Please help.

There seems to be a regression error in

sauce-java / testng / src / main / java / com / saucelabs / testng / SauceOnDemandTestListener.java

Line 133 (markJobAsPassed): I think this should be updates.put("passed", true); not updates.put("passed", false);

(My test status broke after the update to 1.0.9)

It would be good to clean up the folder architecture in the solution. So far I have been storing a lot in Common because that's where things started. However, I don't think that name makes sense. So that package should be broken up into other packages with relevant components.
The unit tests should be moved accordingly so that they can run against their corresponding packages as well.

Clicking on README link for sauce-connect-plugin repo, server gives a 404 response.

http://repository-saucelabs.forge.cloudbees.com/release/com/saucelabs/maven/plugin/sauce-connect-plugin/

We need logging in this method so that we know exactly which object is created as a result of the if statement. We had a bug where some unit tests were not resetting the state back to null of customManager. As a result, some acceptance tests were actually getting a mocked instance of the object and not executing any JavaScript because it was mocked. This would only manifest if some tests ran in a specific order. Making the failure of the tests seem random.

Logging in that method would point out that some test tried to execute and didn't get back the appropriate instance of the JavaScriptInvokerImpl

Sauce Labs Support received this report:

HI, We have both tests that we want to run on our local systems and tests that we want to run with cross browser functionality. As such, we implement our own custom listener which we want to use when on our system, and the SauceLabOnDemandListener when using sauce lab. However, even when we set SELENIUM_IS_LOCAL to True, SauceLabOnDemandListener still is somehow active and causes tests to crash when running on our local machines. We looked into the SauceLab listener and found that in many methods, computation takes place BEFORE the check to see if SELENIUM_IS_LOCAL, which we believe may be the cause for these failures.These methods 'onTestStart(ITestResult result)", "onTestFailure(ITestResult tr)", "onTestSuccess(ITestResult tr)" are performed functionalities before it returns. This cause our CustomListener to crash. Shouldn't it return immediately if isLocal is true without performing any function?

Thank you!

https://mvnrepository.com/artifact/com.saucelabs.maven.plugin/sauce-connect-plugin

The sauce connect version in maven repository is out of date. There are lots of issues in the old version of sauce connect. Can this please package latest sauce connect version?

Thanks.

Inside the JUnit module tests, the WebDriverTest::tearDown method has the following code:

@After
public void tearDown() {
    assertNotNull(driver);
    driver.quit();
}

Putting assertNotNull(driver); inside a tearDown method is not efficient. If the purpose of that assertion is to know if the driver is not null, it is better to throw that exception in a separate test method than to throw it in an @After method.
This patch of code could be refactored to:

@Test
public void assertDriverNotNull(){
    assertNotNull(driver);
}

@After
public void tearDown() {
    if(driver != null){
        driver.quit();
    }
}

Code coverage is currently not working for the project and it should be

A large amount of documentation refers to:

export SAUCE_USERNAME=”myusername”
export SAUCE_ACCESS_KEY=”xxxxxxxx”

The Java client only checks:

    private static final String SAUCE_USER_NAME = "SAUCE_USER_NAME";
    private static final String SAUCE_API_KEY = "SAUCE_API_KEY";

It'd save me a bunch of work if both were supported. If someone attends a presentation, such as jquery uk, the ENV isn't going to work on Java.

This could be something I'm not comprehending, but...

The test suite runs a TestNG test which attempts to use com.saucelabs.testng.WebDriverTest (specified in testng/src/test/resources/xml/testng.xml), which class does not exist. It would be nice to have a clean-running test suite right off the bat.

Hi Ross,

I did a git clone of this project and I am getting compilation errors. I am using IntelliJ 12. Here are the issues-

1. I get cannot resolve symbol simple in SauceBrowserDataProvider
   import org.json.simple.JSONArray;
   import org.json.simple.JSONObject;
   import org.json.simple.parser.JSONParser;
   import org.json.simple.parser.ParseException;

2. Cannot resolve SauceREST in SauceOnDemandTestListener
   import com.saucelabs.rest.SauceREST;

3. Cannot resolve StringUtils in SeleniumRCWithHelperTest
   import org.apache.commons.lang.StringUtils;

When executing tests from our desktop on SauceLabs, we must pass through a corporate firewall (which currently involves a very awkward Selenium custom HttpExecutor configuration).

Could support be added to the SauceOnDemandTestWatcher to support proxies as well? I think we would just need to pass in a Proxy object to SauceREST and use that in the url.openConnection() call.

See http://support.saucelabs.com/tickets/4059

printSessionId is always called.

When I run mvn test, the logs are spammy.

Running appium.training.android.AutomatingASimpleActionTest
test: one
SauceOnDemandSessionID=66011113c5e747af94ad2086fa9fa5de job-name=appium.training.android.AutomatingASimpleActionTest.one
sauce link: https://saucelabs.com/tests/66011113c5e747af94ad2086fa9fa5de

There is no way for me to silence the SauceOnDemandSessionID messages.

When trying to build a maven project that includes the sauce-connect-plugin on JDK9, the build fails with Execution start-sauceconnct of goal com.saucelabs.maven.plugin:sauce-connect-plugin:2.1.23:start-sauceconnect failed: A required class was missing while executing com.saucelabs.maven.plugin:sauce-connect-plugin:2.1.23:start-sauceconnect: javax/xml/bind/DatatypeConverter.

The same build passes just fine on JDK8.

$ java -version
java version "9.0.1"
Java(TM) SE Runtime Environment (build 9.0.1+11)
Java HotSpot(TM) 64-Bit Server VM (build 9.0.1+11, mixed mode)

$ mvn verify
...
[ERROR] Failed to execute goal com.saucelabs.maven.plugin:sauce-connect-plugin:2.1.23:start-sauceconnect (start-sauceconnct) on project sauce-connect-plugin-jdk9: Execution start-sauceconnct of goal com.saucelabs.maven.plugin:sauce-connect-plugin:2.1.23:start-sauceconnect failed: A required class was missing while executing com.saucelabs.maven.plugin:sauce-connect-plugin:2.1.23:start-sauceconnect: javax/xml/bind/DatatypeConverter
[ERROR] -----------------------------------------------------
[ERROR] realm =    plugin>com.saucelabs.maven.plugin:sauce-connect-plugin:2.1.23
[ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy
[ERROR] urls[0] = file:/Users/viktor/.m2/repository/com/saucelabs/maven/plugin/sauce-connect-plugin/2.1.23/sauce-connect-plugin-2.1.23.jar
[ERROR] urls[1] = file:/Users/viktor/.m2/repository/com/saucelabs/ci-sauce/1.129/ci-sauce-1.129.jar
[ERROR] urls[2] = file:/Users/viktor/.m2/repository/org/codehaus/plexus/plexus-archiver/3.4/plexus-archiver-3.4.jar
[ERROR] urls[3] = file:/Users/viktor/.m2/repository/org/codehaus/plexus/plexus-utils/3.0.24/plexus-utils-3.0.24.jar
[ERROR] urls[4] = file:/Users/viktor/.m2/repository/org/codehaus/plexus/plexus-io/2.7.1/plexus-io-2.7.1.jar
[ERROR] urls[5] = file:/Users/viktor/.m2/repository/org/apache/commons/commons-compress/1.11/commons-compress-1.11.jar
[ERROR] urls[6] = file:/Users/viktor/.m2/repository/org/iq80/snappy/snappy/0.4/snappy-0.4.jar
[ERROR] urls[7] = file:/Users/viktor/.m2/repository/org/tukaani/xz/1.5/xz-1.5.jar
[ERROR] urls[8] = file:/Users/viktor/.m2/repository/commons-lang/commons-lang/2.6/commons-lang-2.6.jar
[ERROR] urls[9] = file:/Users/viktor/.m2/repository/commons-cli/commons-cli/1.3.1/commons-cli-1.3.1.jar
[ERROR] urls[10] = file:/Users/viktor/.m2/repository/org/json/json/20160212/json-20160212.jar
[ERROR] urls[11] = file:/Users/viktor/.m2/repository/com/google/code/findbugs/jsr305/3.0.1/jsr305-3.0.1.jar
[ERROR] urls[12] = file:/Users/viktor/.m2/repository/org/apache/xbean/xbean-reflect/3.4/xbean-reflect-3.4.jar
[ERROR] urls[13] = file:/Users/viktor/.m2/repository/log4j/log4j/1.2.12/log4j-1.2.12.jar
[ERROR] urls[14] = file:/Users/viktor/.m2/repository/commons-logging/commons-logging-api/1.1/commons-logging-api-1.1.jar
[ERROR] urls[15] = file:/Users/viktor/.m2/repository/com/google/collections/google-collections/1.0/google-collections-1.0.jar
[ERROR] urls[16] = file:/Users/viktor/.m2/repository/junit/junit/3.8.2/junit-3.8.2.jar
[ERROR] urls[17] = file:/Users/viktor/.m2/repository/org/apache/commons/commons-lang3/3.4/commons-lang3-3.4.jar
[ERROR] urls[18] = file:/Users/viktor/.m2/repository/javax/enterprise/cdi-api/1.0/cdi-api-1.0.jar
[ERROR] urls[19] = file:/Users/viktor/.m2/repository/org/eclipse/sisu/org.eclipse.sisu.inject/0.3.2/org.eclipse.sisu.inject-0.3.2.jar
[ERROR] urls[20] = file:/Users/viktor/.m2/repository/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar
[ERROR] urls[21] = file:/Users/viktor/.m2/repository/com/saucelabs/sauce-connect/3.1.32/sauce-connect-3.1.32.jar
[ERROR] urls[22] = file:/Users/viktor/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar
[ERROR] urls[23] = file:/Users/viktor/.m2/repository/com/saucelabs/saucerest/1.0.32/saucerest-1.0.32.jar
[ERROR] urls[24] = file:/Users/viktor/.m2/repository/com/googlecode/json-simple/json-simple/1.1.1/json-simple-1.1.1.jar
[ERROR] urls[25] = file:/Users/viktor/.m2/repository/org/apache/httpcomponents/httpclient/4.5/httpclient-4.5.jar
[ERROR] urls[26] = file:/Users/viktor/.m2/repository/org/apache/httpcomponents/httpcore/4.4.1/httpcore-4.4.1.jar
[ERROR] urls[27] = file:/Users/viktor/.m2/repository/commons-logging/commons-logging/1.2/commons-logging-1.2.jar
[ERROR] urls[28] = file:/Users/viktor/.m2/repository/commons-codec/commons-codec/1.9/commons-codec-1.9.jar
[ERROR] Number of foreign imports: 1
[ERROR] import: Entry[import  from realm ClassRealm[maven.api, parent: null]]
[ERROR]
[ERROR] -----------------------------------------------------
[ERROR] : javax.xml.bind.DatatypeConverter
...

The full source of the project is attached:
sauce-connect-plugin-jdk9.zip.

To reproduce the issue download and extract the source code, then run mvn verify inside it. The expected result is that the sauce connect plugin starts and complains that the
Sauce Labs username is not set:

Nov 20, 2017 2:44:57 PM com.saucelabs.saucerest.SauceREST retrieveResults
SEVERE: Error retrieving Sauce Results
java.io.IOException: Server returned HTTP response code: 401 for URL: https://saucelabs.com/rest/v1/YOUR_SAUCE_USERNAME/tunnels

Not sure why my session id is coming back as null but I do believe it has to do with the fact that I am running my testng tests in parallel with 22 different threads. I am seeing the following error in the jenkins console when running my test classes using maven. None of the test results are shown in the sauce labs UI as passed or failed, only "?" and complete. I am using the listener in my base test class as shown here:

public class TestNGTestBase implements SauceOnDemandSessionIdProvider, SauceOnDemandAuthenticationProvider {

All my test classes extend my baseclass called TestNGTestBase and contain the annotation for adding the SauceOnDemandTestListener:

@listeners({SauceOnDemandTestListener.class})
public class AcceptHintFromAllHintsPage extends TestNGTestBase {

The Jenkins console shows the following error for each test class that is executed:
SauceOnDemandSessionID=null job-name=AcceptHintFromAllHintsPage
May 12, 2016 12:20:57 PM com.saucelabs.saucerest.SauceREST closeInputStream
WARNING: Error closing result stream
java.io.FileNotFoundException: https://saucelabs.com/rest/v1/mldean/jobs/null
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1836)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
at com.saucelabs.saucerest.SauceREST.closeInputStream(SauceREST.java:442)
at com.saucelabs.saucerest.SauceREST.updateJobInfo(SauceREST.java:410)
at com.saucelabs.testng.SauceOnDemandTestListener.markJobStatus(SauceOnDemandTestListener.java:115)
at com.saucelabs.testng.SauceOnDemandTestListener.onTestSuccess(SauceOnDemandTestListener.java:142)
at org.testng.internal.Invoker.runTestListeners(Invoker.java:1899)
at org.testng.internal.Invoker.runTestListeners(Invoker.java:1879)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:778)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Codacy is our static analysis tool that I enabled with default configuration. This might be good enough. However, it might also require some tweaking of the types of rules that we want it to check for in Java. Some rules might be silly. Others might not be turned on and should be. I am not a Java expert and would like assistance from a more experienced Engineer to see which rules, if any, should be turned on/off. That way, our static code analysis can be most relevant

Hi,

Here is my maven configuration , The sauce connect is not starting while maven pre-integration-test phase. Instead directly executing the tests.

<properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>
    <pluginRepositories>
    <pluginRepository>
       <id>saucelabs-repository</id>
       <url>http://repository-saucelabs.forge.cloudbees.com/release</url>
       <releases>
           <enabled>true</enabled>
       </releases>
       <snapshots>
           <enabled>true</enabled>
       </snapshots>
    </pluginRepository>
    </pluginRepositories>
    <!-- dependencies -->
    <dependencies>
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>6.9.9</version>
        </dependency>
        <dependency>
            <groupId>com.saucelabs</groupId>
            <artifactId>sauce_testng</artifactId>
            <version>2.1.21</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>2.52.0</version>
        </dependency>
        <dependency>
            <groupId>com.saucelabs</groupId>
            <artifactId>sauce_java_common</artifactId>
            <version>2.1.18</version>
        </dependency>
        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>1.2.17</version>
        </dependency>
        <dependency>
            <groupId>javax.mail</groupId>
            <artifactId>mail</artifactId>
            <version>1.4</version>
        </dependency>
        <dependency>
            <groupId>org.apache.poi</groupId>
            <artifactId>poi</artifactId>
            <version>3.10-FINAL</version>
        </dependency>
        <dependency>
            <groupId>org.apache.poi</groupId>
            <artifactId>poi-ooxml</artifactId>
            <version>3.10-FINAL</version>
        </dependency>
        <dependency>
            <groupId>xml-apis</groupId>
            <artifactId>xml-apis</artifactId>
            <version>1.4.01</version>
        </dependency>
    </dependencies>
<!-- Build execution management -->
    <build>
        <plugins>
            <!-- Include Sauce Connect plugin -->
            <plugin>
                    <groupId>com.saucelabs.maven.plugin</groupId>
                    <artifactId>sauce-connect-plugin</artifactId>
                    <version>2.1.3</version>
                    <configuration>
                            <sauceUsername>SEC_ID</sauceUsername>
                            <sauceAccessKey>SEC_KEY</sauceAccessKey>
                            <!--<options>-i testing</options>-->
                    </configuration>
                    <executions>
                            <execution>
                                    <id>start-sauceconnect</id>
                                    <phase>pre-integration-test</phase>
                                    <goals>
                                            <goal>start-sauceconnect</goal>
                                    </goals>
                            </execution>
                            <execution>
                                    <id>stop-sauceconnect</id>
                                    <phase>post-integration-test</phase>
                                    <goals>
                                            <goal>stop-sauceconnect</goal>
                                    </goals>
                            </execution>
                    </executions>
            </plugin>

            <plugin>
         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-failsafe-plugin</artifactId>
         <version>2.19.1</version>
                 <configuration>
          <suiteXmlFiles>
            <suiteXmlFile>testng.xml</suiteXmlFile>
          </suiteXmlFiles>
        </configuration>
         <executions>
           <execution>
             <id>integration-test</id>
             <goals>
               <goal>integration-test</goal>
                             <goal>verify</goal>
             </goals>
           </execution>
         </executions>
       </plugin>

        </plugins>
    </build>

I have found that running a TestNG data-driven test using SauceOnDemandTestListener.class does not work correctly. For a data-driven test, one @test is run several times for the number of rows in a CSV as an example. Using SauceOnDemandTestListener, the Sauce Labs job is marked as pass if a row of test data passes, and marked as failed if a row of test data fails. So the entire job passes if the LAST row of test data passes, and fails if the LAST row of test data fails.

I actually ended up adding to the SauceOnDemandTestListener. I added a boolean called passed to the class and set it to true by default.

private boolean passed = true;

I then modified onTestFailure method by adding the following if logic:

if (this.passed) { this.passed = false; }

And finally modified onTestSuccess with:

if (this.passed) { markJobAsPassed(); }

That way, if a failure happens, the job will never be marked as passed in subsequent tests for a data-driven test. May not be the best way to handle the issue, but it's worked for me so far.

It is not possible to add two options configurations

not working:
<configuration> ... <options>-p testproxy:3128</options> <options>-i local_test</options> ... </configuration>

working:
<configuration> ... <options>-p testproxy:3128</options> ... </configuration>

or

<configuration> ... <options>-i local_test</options> ... </configuration>

Is it possible to fix this bug or is this an expected behaviour?

Tried running a test SampleSauceTest with ConcurrentParameterized.class on Sauce lab is throwing initialization error. Could you please help me on fixing this issue

I want to add a configuration option to execute local test runs by re-using SauceOnDemandTestListener::SELENIUM_IS_LOCAL variable, which skips initialization of sauceRest when the runtime property variable is set to true.

SauceOnDemandTestListener's onTestStart and onTestFailure both have null checks prior to de-referencing SauceOnDemandSessionIdProvider::getSessionId, but onTestSuccess does not.

This leads to NPEs with the following stack trace, one for each successful test:

SauceOnDemandSessionID=null job-name=testThree
java.lang.NullPointerException
        at com.saucelabs.testng.SauceOnDemandTestListener.markJobStatus(SauceOnDemandTestListener.java:115)
        at com.saucelabs.testng.SauceOnDemandTestListener.onTestSuccess(SauceOnDemandTestListener.java:142)
        at org.testng.internal.TestListenerHelper.runTestListeners(TestListenerHelper.java:70)
        at org.testng.internal.Invoker.runTestListeners(Invoker.java:1379)
        at org.testng.internal.Invoker.invokeMethod(Invoker.java:624)
        at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:707)
        at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:979)
        at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
        at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

SauceOnDemandTestListener::onTestSuccess should have null checks, or perhaps better yet, skip their RPCs if isLocal property is true.

The JUnit/TestNG archetypes need some love, care, and documentation.

UPDATE: Instead of "fixing" these archetypes, we're going to create archetypes for our other sample Java frameworks.

A/C:
- Update the README with instructions for how to use the archetypes
- The pom.xml lists the incorrect default artifactId (there's a typo, missing a curly brace)
- Update the default version ID to 1.0.0-SNAPSHOT
- Clean up and bring all Java files into compliance with modern Java practices
- Ensure Codacy report is clean

1. Create project that uses this plugin for integration tests with pre & post phase bindings
2. Run the "integration-test" multiple times (i..e mvn clean integration-test -DtestGroup=A && mvn clean integration-test -DtestGroup=B)
3. Watch as all subseqnet tests will fail when trying to open the tunnel (previous invocation did not properly shutdown tunnel because post-integration-test is never reached, but I would still expect the process to be able to recover by purging old pid and starting new, or leveraging existing tunnel)

In any case, the error is not helpful

Exception occurred retrieving tunnel information
org.json.JSONException: JSONObject["tunnel_identifier"] not a string.

[INFO] --- sauce-connect-plugin:2.1.23:start-sauceconnect (start-sauceconnct) @ statuspage-banner-atlassian-server --- [INFO] Starting Sauce Connect May 03, 2017 5:44:56 PM com.saucelabs.ci.sauceconnect.AbstractSauceTunnelManager activeTunnelIdentifier WARNING: Exception occurred retrieving tunnel information org.json.JSONException: JSONObject["tunnel_identifier"] not a string. at org.json.JSONObject.getString(JSONObject.java:721) at com.saucelabs.ci.sauceconnect.AbstractSauceTunnelManager.activeTunnelIdentifier(AbstractSauceTunnelManager.java:440) at com.saucelabs.ci.sauceconnect.AbstractSauceTunnelManager.openConnection(AbstractSauceTunnelManager.java:289) at com.saucelabs.maven.StartSauceConnectMojo.execute(StartSauceConnectMojo.java:73) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:133) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:108) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:76) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:116) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:361) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155) at org.apache.maven.cli.MavenCli.execute(MavenCli.java:584) at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:213) at org.apache.maven.cli.MavenCli.main(MavenCli.java:157) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) May 03, 2017 5:44:56 PM com.saucelabs.ci.sauceconnect.AbstractSauceTunnelManager logMessage INFO: /root/sc-4.4.2-linux/bin/sc already exists, so not extracting May 03, 2017 5:44:56 PM com.saucelabs.ci.sauceconnect.SauceConnectFourManager prepAndCreateProcess INFO: Launching Sauce Connect 4.4.2 [/root/sc-4.4.2-linux/bin/sc, -u, eddiewebb, -k, 1e242a24-66c9-4326-8bcd-f917dad39073, -P, 4445] 3 May 17:44:56 - Sauce Connect 4.4.2, build 3154 c8dd102-dirty 3 May 17:44:56 - Using CA certificate bundle /etc/ssl/certs/ca-certificates.crt. 3 May 17:44:56 - Using CA certificate verify path /etc/ssl/certs. May 03, 2017 5:44:56 PM com.saucelabs.ci.sauceconnect.AbstractSauceTunnelManager$StreamGobbler processLine INFO: 3 May 17:44:56 - Sauce Connect 4.4.2, build 3154 c8dd102-dirty May 03, 2017 5:44:56 PM com.saucelabs.ci.sauceconnect.AbstractSauceTunnelManager$StreamGobbler processLine INFO: 3 May 17:44:56 - Using CA certificate bundle /etc/ssl/certs/ca-certificates.crt. 3 May 17:44:56 - Error creating pidfile /tmp/sc_client.pid. May 03, 2017 5:44:56 PM com.saucelabs.ci.sauceconnect.AbstractSauceTunnelManager$StreamGobbler processLine INFO: 3 May 17:44:56 - Using CA certificate verify path /etc/ssl/certs. May 03, 2017 5:44:56 PM com.saucelabs.ci.sauceconnect.AbstractSauceTunnelManager$StreamGobbler processLine INFO: 3 May 17:44:56 - Error creating pidfile /tmp/sc_client.pid. May 03, 2017 5:44:56 PM com.saucelabs.ci.sauceconnect.AbstractSauceTunnelManager$StreamGobbler processLine INFO: ERROR: can't lock pidfile /tmp/sc_client.pid. ERROR: can't lock pidfile /tmp/sc_client.pid. May 03, 2017 5:44:56 PM com.saucelabs.ci.sauceconnect.AbstractSauceTunnelManager logMessage INFO: Sauce Connect can't lock pidfile, attempting to close open Sauce Connect processes [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 15.692 s [INFO] Finished at: 2017-05-03T17:44:56+00:00 [INFO] Final Memory: 61M/475M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal com.saucelabs.maven.plugin:sauce-connect-plugin:2.1.23:start-sauceconnect (start-sauceconnct) on project statuspage-banner-atlassian-server: Execution start-sauceconnct of goal com.saucelabs.maven.plugin:sauce-connect-plugin:2.1.23:start-sauceconnect failed. NullPointerException -> [Help 1] [ERROR]