openliberty / ci.ant Goto Github PK

Add a task for installing features (esa files).

https://github.com/WASdev/ci.ant/blob/master/src/main/java/net/wasdev/wlp/ant/InstallFeatureTask.java#L70 doesn't handle name="featureA,featureB" which was possible when it was implemented to use featureManager. The Gradle plugin still makes use of the name attribute and passes multiple features in this way. Also, old ant scripts using name must also be broken.

When I add outputDir to a server create ant task it is used as config dir and output dir ends with appended serverName.

I discovered this while trying to build a Gradle plugin using the Ant tasks.

This project illustrates the process: https://github.com/corneil/wasdev-wlp-test

Console output:

invoke:[operation:create, serverName:test, installDir:build\wlp]
[ant:antlib:net.wasdev.wlp.ant:server] CWWKM2001I: server.config.dir is build\wlp\usr\servers\test.
[ant:antlib:net.wasdev.wlp.ant:server] CWWKM2001I: server.output.dir is build\wlp\usr\servers\test.
[ant:antlib:net.wasdev.wlp.ant:server] CWWKM2001I: Invoke command is ["build\wlp\bin\server.bat", create, test].
[ant:antlib:net.wasdev.wlp.ant:server] Server test created.

With outputDir added with default value when not supplied.

invoke:[operation:create, serverName:test, installDir:build\wlp, outputDir:build\wlp\usr\servers\test]
[ant:antlib:net.wasdev.wlp.ant:server] CWWKM2001I: server.config.dir is build\wlp\usr\servers\test.
[ant:antlib:net.wasdev.wlp.ant:server] CWWKM2001I: server.output.dir is build\wlp\usr\servers\test\test.
[ant:antlib:net.wasdev.wlp.ant:server] CWWKM2001I: Invoke command is ["build\wlp\bin\server.bat", create, test].
[ant:antlib:net.wasdev.wlp.ant:server] CWWKE0005E: The runtime environment could not be launched.
[ant:antlib:net.wasdev.wlp.ant:server] CWWKE0045E: It was not possible to create the server called test because the server directory build\wlp\usr\servers\test already exists.

The folder doesn't exist before createServerBroken is executed because there is a check to skip the task if the server folder exists.

Provide a way to create a server from an existing template (e.g. same template as in ./bin/server create --template

While I was using the ant plugin I found some scenarios where the "deploy" task seems to be working weird.

First scenario:

When you are trying to deploy a single application using the "file" parameter of the <wlp:deploy/> task if the file set as parameter does not exist, the deploy is not done but the build ends as successful, I think this should not be happening. It should return a BuildException since the task never performed its work.

This are the configurations needed to reproduce the scenario.

File: build.xml

<wlp:deploy ref="idMyServer" file="a">

In this case the file "a" does not exist in my project but even when the deploy task is not "deploying" this file the result of this build is successful.

The console out was this:

I mean I don't know if this is the way how this task is supposed to work, but I think it should be better if in this case the build could fail because the file does not exist.

Second scenario:

If I use a fileset to select which files I want to deploy and I set an existing file only, the application is deployed correctly, but, if I set two files, one existing and the other one not, the deploy is never done. Even if one of the files actually exists.

In this case I set two files:

• test-war.war (This file actually exists on my resources folder.)
• noexists.war (This file does not exist.)

The configurations to reproduce this scenario are:

File: build.xml

<property name="resources_dir" value="C:/Users/Administrator/Documents/AntPractices/resources"/>

<wlp:deploy ref="idMyServer">
    <fileset dir="${resources_dir}" casesensitive="no">
        <filename name="test-war.war" />
        <filename name="noexists.war" />
    </fileset>
</wlp:deploy>

The console output was this:

As you can see the output is the same in both scenarios. If I only deploy the existing file the deploy is done correctly.

Third scenario:

If the file parameter of the <wlp:deploy/> task is set as "" (empty), the deploy task try to deploys the project's folder. (In my case "AntPractices")

File: build.xml

<wlp:deploy ref="idMyServer" file=""/>

The console output was this:

In this case the server is stopped but I think it would be better if it could throw an specific BuildException to avoid try to deploy the entire folder.

The structure of my proyect is:

-AntPractices
    -resources/
        * test-war.war (file)
    wlp-anttasks-1.1-SNAPSHOT.jar
    build.xml

The process builder in AbstractTask doesn't redirect System Error to System Out, meaning that failures can be missed if not record is logged to System Out.

It would be helpful to implement a task in ant as described in OpenLiberty/ci.maven#9 so we can clean the droppins dir if necessary

Add a parameter to the deploy task that defines the final file-name of the app to be deployed. It would help to solve the issue OpenLiberty/ci.maven#28

The following message appears repeatedly while waiting for the server to start. Optimize the logic to only print the message once.

CWWKM2013I: The file {0} being validated does not exist.

This is logged in AbstractTask.findStringsInFileCommon() method, which is called by findStringInFile(), which is called by waitForStringInLog() in a loop based on the serverStartTimeout value.

When running installLiberty task I get the following error:

Execution failed for task ':starter-microservice-persistence:installLiberty'.

No candidates found for 8.5.+ version.

I am running this via the ci.gradle plugin using the config: type = "javaee7"

Enhance the install-liberty task with ability to install from the different .zip files.

Consider setting an appropriate timeout for the job and/or steps in the github action maven.yml file.

https://docs.github.com/en/free-pro-team@latest/actions/reference/workflow-syntax-for-github-actions#jobsjob_idstepstimeout-minutes

Following the issue 46 of the Maven project (OpenLiberty/ci.maven#46), when the acceptLicence parameter is not set or set to false, the tasks freezes before the "Select [1] I Agree, or [2] I do not Agree:" section, so the user can't accept nor decline the license.

Steps to reproduce

• Uninstall mongodb-2.0 feature
• build.xml:

The execution will stop at this point...

While executing directly the featureManager gives the user the option to accept or decline:

Add support for the "run" server operation - for running the server in foreground.

Adding the --os parameter could give the user more control about how they want to create the package of the server and also reduce the size of the resulting archive.

For more information, see: http://www-01.ibm.com/support/knowledgecenter/SSEQTP_8.5.5/com.ibm.websphere.wlp.doc/ae/twlp_setup_package_server.html?cp=SSEQTP_8.5.5%2F1-3-11-0-3-2-1-6

It would be nice to deploy applications to a remotely running liberty server

Improve the server task to support the following:

1. support dump and javadump operations
2. "include" attribute for dump, javadump, and package operations
3. support "template" attribute for create operation
4. "resultProperty" for status operation.
5. starting a started server is not an error
6. stopping a stopped server is not an error
7. "clean" attribute defaults to false.

Add task for installing Liberty from WASdev repository or using Liberty runtime.jar archive.

The WebSphere Liberty 20.0.0.9 integration tests have been failing consistently on Windows due to the following error during the clean task. Note that 20.0.0.6 and earlier did not get this error, and also Open Liberty does not get this error on any version or any platform.

[INFO] around Ant part ...... @ 4:147 in D:\a\ci.ant\ci.ant\target\it\tests\deploy-war-it\target\antrun\build-main.xml
[INFO] at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
[INFO] at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
[INFO] at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
[INFO] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
[INFO] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
[INFO] at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
[INFO] at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
[INFO] at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
[INFO] at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
[INFO] at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
[INFO] at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
[INFO] at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
[INFO] at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
[INFO] at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
[INFO] at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
[INFO] at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
[INFO] at java.lang.reflect.Method.invoke (Method.java:498)
[INFO] at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
[INFO] at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
[INFO] at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
[INFO] at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[INFO] Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException has occured: The following error occurred while executing this line:
[INFO] D:\a\ci.ant\ci.ant\target\it\tests\deploy-war-it\build.xml:34: CWWKM2031E: Cannot delete file D:\a\ci.ant\ci.ant\target\it\tests\install-server-it\target\wlp_output\deploy.war\workarea.sLock.

Here is a build with the failure: https://github.com/OpenLiberty/ci.ant/runs/1425836093?check_suite_focus=true#step:5:450

Such as: installing from an alternative repository; point to a server dir and install-feature should install all the necessary features (dependencies included - just like installUtility)

Liberty versioning has changed and so we need to update the Liberty version range to 16.+ to select the latest version available.

When waiting for an updated message in a log file, check the messages based on timestamp instead of counting the number of previous occurrences of that message.

Right now, to install features we have to add them one by one using the name parameter in the install-feature task like this:

<wlp:install-feature installDir="${wlp_install_dir}" name="mongodb-2.0" 
    whenFileExists="ignore" acceptLicense="true"/>

It would be cleaner and more readable to specify the features in nested elements like this:

<wlp:install-feature installDir="${wlp_install_dir}" whenFileExists="ignore" acceptLicense="true">
    <feature>mongodb-2.0</feature>
    <feature>oauth-2.0</feature>
</wlp:install-feature>

When a server stop command has a return code >= 20, it means an exception occurred during the stop. However, the server may still have stopped. One of the conditions that can cause a return code >= 20 is if the stop takes longer than 30 seconds. In that case, we will fail the build, even though the server may have stopped successfully.

When a return code >= 20 is received, we should instead issue a server status command. If the server status return code = 1, then we can presume the server stopped and return without error. If the server status return code = 0, the server is still running and we should fail the build as we did previously.

This is related to issue OpenLiberty/ci.gradle#554.

It would be nice if the integration tests could be performed in the travis build. And also, add support to execute the integration tests by downloading a wlp automatically by just providing the version and the license code.

Currently the test runtime is wlp-runtime, but this will be removed as of 18.0.0.2.

Right now, the code does not check if the WLP_USER_DIR and WLP_OUTPUT_DIR environment properties were set - that is, were inherited by the Java process. And that sometimes may cause server to output files to one location while the task code is looking at another.
The questions is whether the Ant task should respect these env. properties or not.

See OpenLiberty/ci.maven#468

We only test against v8557. v8558 is available now. Brings into question - how many versions should we test against. I think current and previous is good enough.

It should be possible to compile JSPs at build time so it isn't done at runtime on first request.

It would be nice to create a task to perform some kind of clean over the server. For example, cleaning up some logs, or other operations to enhance liberty.

I was looking around for how to automate our WebSphere Liberty related tasks and I first found IBM's repository with wlp-anttasks and then yours. Is your repository and wlp-anttasks related to IBM's or is this an independent effort?

During server create from the maven plugin, I see this:

INFO CWWKM2001I: Invoke command is [C:\Users\IBM_ADMIN\Downloads\libertyProject(1)\myProject-wlpcfg\target\wlp\bin\server.bat, create, LibertyProjectServer].
INFO 'C:\Users\IBM_ADMIN\Downloads\libertyProject' is not recognized as an internal or external command,
INFO operable program or batch file.

Looks like the paths need some special character handling.

The Liberty Maven plugin should have a task to install (unzip) the liberty zip freemium editions currently located on the DHE maven repo, and eventually on maven central.

There are shared parameters for all the tasks. It would be fine to create a file that contain them and do not to have them repeated in all the tasks description.

Hi,
We have been using the Maven plugin, which uses this Ant plugin for shell/OS interaction, and are experiencing some problems running under (Git) Bash on Windows.
The behavior is that on all Windows shells, the StartServer Mojo, for example, will run the "server.bat" script.
We think it would be reasonable (and desirable) when running under Bash, regardless if it is under Windows, that the corresponding "server", etc. scripts be run, and not the ones with the ".bat" suffix.
Thank you in advance!
-Mike

Hi guys, it seems that when you declare a server configuration with an id (I mean, to be reused in future) you have to set an operation by default, this seems unhelpful because this predefined operation would be executed when the build file is kicked off.

For example, If I want to set the following server definition as reference:

    <wlp:server id="idMyServer"
                installDir="${wlp_install_dir}"
                userDir="C:/Users/Administrator/Documents/AntPractices/testDir"
                outputDir="C:/Users/Administrator/Documents/AntPractices/testDir/output"
                serverName="MyServer"
                operation="status" />

So, if I try to use this ref with another operation instead to be replaced are executed both operations.

    <wlp:server ref="idMyServer" operation="start"/>

The console output is:

As you can see is being executed the operation set as default in the server definition plus the operation set when is used the server reference. It would be better if the second operation parameter could overwrite the first one. Another solution could be avoid require an operation in the server definition and let the user set the operation when the ref is used.

I mean, something like this:

    <wlp:server id="idMyServer"
                installDir="${wlp_install_dir}"
                userDir="C:/Users/Administrator/Documents/AntPractices/testDir"
                outputDir="C:/Users/Administrator/Documents/AntPractices/testDir/output"
                serverName="MyServer"/>

   <wlp:server ref="idMyServer" operation="start"/>

In this case only the start would be executed.

After installation of a feature that is already installed in liberty with the install-feature task, it returns rc=22 interrupting the build process.

Steps to reproduce

• In a build.xml file configure the ant tasks
• Set a target with the following configuration:
  <wlp:install-feature installDir="${liberty.dir}" acceptLicense="true" whenFileExists="replace" name="jaxrs-1.1"/>
• Execute the target.

Expected result

Desire result
Notify the user that the feature is already installed but continue the build.

Comments
The --when-file-exists parameter is used to decide what happens when installing a feature if a bundle already exist, it's not used to define what happens when a feature is already installed.

The deploy and the undeploy task will stop the server if a failure occurs deploying or undeploying the application. That should not happen.

It would be nice to have a clearer structure of the ci.ant plugin documentation by dividing each section in different .md files.

Running the following ant task on my liberty installation returns the following ouptut:

Ant properties:

<property name="wlp_install_dir" value="D:/EnvtDev/liberty_8558"/>
<property name="serverName" value="defaultServer"/>
<property name="wlp_output" value="D:/EnvtDev/liberty_8558/usr/servers/"/>
<property name="wlp_usr" value=""/>

Ant task:

<wlp:server id="WinSights" installDir="${wlp_install_dir}" 
           userDir="${wlp_usr}" outputDir="${wlp_output}" serverName="${serverName}" 
		operation="status"/>

Output:

[wlp:server] CWWKM2001I: server.config.dir is D:\EnvtDev\git\winsights\doc\servers\defaultServer.
[wlp:server] CWWKM2001I: server.output.dir is D:\EnvtDev\liberty_8558\usr\servers\defaultServer.
[wlp:server] CWWKM2001I: Invoke command is ["D:\EnvtDev\liberty_8558\bin\server.bat", status, defaultServer].
[wlp:server] CWWKE0005E: The runtime environment could not be launched.
[wlp:server] CWWKE0031E: Specified server defaultServer does not exist; use the --create option to create a new server. serverPath: D:\EnvtDev\git\winsights\doc\servers\defaultServer
BUILD SUCCESSFUL
Total time: 914 milliseconds

I guess the task fails because server.config.dir is not properly set. How should this variable be set up.

There is not much examples in the doc.

Add a task to display the product information properties of the runtime and product extensions. This will be useful for retrieving information such as the product ID and version.

Solution for OpenLiberty/ci.maven#1151

The issue described in OpenLiberty/ci.maven#279 needs to be fixed in ci.ant.

I have tried to change the logging directory (in the server.xml file). The plugin has not "seen" the server start.

It seems the Liberty plugin try to locate the "started message" in the default location.

[INFO] CWWKM2010I: Recherche de CWWKF0011I dans D:\Workspace\ZUDB0\RSA_9_Developer\SRVT_RecupererListeCartesPaylib-TEST\target\liberty\wlp\usr\servers\defaultSe
rver\logs\messages.log. Cette recherche arrive à expiration après 30 secondes.

The problem is down in the AbstractTask.getLogFile() method where we use the server output directory and do not consider logDirectory attribute from the logging element such as:
<logging logDirectory="/some/log/directory">

When you use the deploy/undeploy tasks if the dropins monitoring is disabled the tasks will fail because they expect to see a message in the log when the application is running. If dropins monitoring is disabled I think the deploy should work. Either the server should be restarted, or it should expect the calling ant script to deal with that, perhaps with an log message being written. Or something else I didn't write in here.

Hi guys, I was wondering if is possible to add the ability to uninstall features from ant, currently only "install" is supported, so it would be awesome if we could uninstall features too.

Right now applications are deployed/undeployed using the dropins directory. However, it is more preferable to deploy/undeploy the applications by updating the server.xml and adding/removing the element.