ibm-datapower / ertool

Error Report Tool (ERTool)

ERTool is a java package for data analysis which simplifies processing large data-sets. This primarily focuses on the multipart-data obtained in a DataPower Error Report (.txt.gz), MQ Appliance (txt.gz) through Analytics.xml. Additionally supports Cast Iron/API Management (.tar.gz formatted files/directory lists) through postmortem.xml.

Prerequisites

• JDK 11 or later
• Apache Ant (Tested with 1.10.7)

Building

After you obtain the source either via a Git client or by downloading the repository zip file, you can build the code using Apache Ant. Enter the framework directory and issue the command below (assumes Apache Ant is in the path):

ant -buildfile build.xml

The ErrorReport.jar will be found in dist/.

Running

For GUI:

java -jar ErrorReport.jar -gui

Special note regarding running the GUI on macOS: You will need to specify -XstartOnFirstThread in the vmargs:

java -XstartOnFirstThread -jar ErrorReport.jar -gui

For Text/CLI with no logging:

java -jar ErrorReport.jar -file "error-report.txt.gz" -analyticsfile Analytics.xml -loglevel none 1> "destinationfile.txt"

For Text/CLI with logging:

java -jar ErrorReport.jar -file "error-report.txt.gz" -analyticsfile Analytics.xml -loglevel info -outfile "destinationfile.txt"

• In the CLI the -format HTML argument can be used to generate an HTML report to the destination file.

• If ErrorReport.jar runs out of memory, increase the available JVM by adding -Xmx8096m as an argument to java.
• If Stack Overflow Exceptions occur increase the stack size of the JVM by adding -Xss8m as an argument to java.
• In cases of Stack Overflow some formulas will require large stack sizes, in testing some have required 512M for 100K lines parsed in regular expressions.
• The loglevel argument can be set to none|info|debug to provide more information when testing formulas.

Contributing

If you want to contribute to the project, you will need to fill in the appropriate Contributor License agreement which can be found under the CLA directory. Follow the directions inside the document so that any submissions can be properly accepted into the repository.

License

The code is licensed under the terms of the Apache License 2.0. See the acompanying 'LICENSE' file for further details.

ertool's Issues

old versions of DataPower do not always generate an out of support error

i have a XI52.4.0.2.0 and do not get the out of support error

ERTool not running on Windows 32 bit JVM

I have 64 bit Windows 7, with 32 bit JVM (jdk1.8.0_141).

I compiled the ERTool successfully, but when I try to run the jar it is giving following error:

Exception in thread "main" java.lang.UnsatisfiedLinkError: Cannot load 64-bit SWT libraries on 32-bit JVM

ant build fails on macOS

Java compilation error in AnalyticsProcessor.java:365

sh-3.2$ ant
Buildfile: /Users/xx/Document/GitHub/ertool/framework/build.xml

init:
[mkdir] Created dir: /Users/xx/Document/GitHub/ertool/framework/bin
[mkdir] Created dir: /Users/xx/Document/GitHub/ertool/framework/bin/tools

compile:
[javac] /Users/xx/Document/GitHub/ertool/framework/build.xml:62: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
[javac] Compiling 82 source files to /Users/xx/Document/GitHub/ertool/framework/bin
[javac] warning: [options] bootstrap class path not set in conjunction with -source 1.7
[javac] /Users/xx/Document/GitHub/ertool/framework/src/com/ibm/datapower/er/Analytics/AnalyticsProcessor.java:365: error: ')' expected
[javac] if ( ordField.getFieldValue().startsWith(("{Condition:"))
[javac] ^
[javac] 1 error
[javac] 1 warning

BUILD FAILED
/Users/xx/Document/GitHub/ertool/framework/build.xml:62: Compile failed; see the compiler error output for details.

reference to libswt is static to the win-32 one

Hey

It seems that you refer to libswt-win32.so
Is there a reason that you refer to an environment specific swt library ?
On windows it works like a charm (nice tool btw) on linux it doesn't.

ERTool Analytics Mapped Table Formatting for text format is not ideal

The mapped table formatting introduced by 9b6eb5d needs attention to resolve text formatting only tabbing. Needs proper tabbing/spacing dependent on size of the column values (likely to use system.format).

MacBook arm M1 issue

Hello,

on Mac book - M1 arm there is issue with SWT library.
Tried on intel Mac, on both same java version, intel works, arm not

andrej@Andrejs-MacBook-Pro dist % java --version java 11.0.16 2022-07-19 LTS Java(TM) SE Runtime Environment 18.9 (build 11.0.16+11-LTS-199) Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.16+11-LTS-199, mixed mode)

andrej@Andrejs-MacBook-Pro dist % java -XstartOnFirstThread -jar ErrorReport.jar -gui
Exception in thread "main" java.lang.UnsatisfiedLinkError: Could not load SWT library. Reasons: 
	no swt-cocoa-4948r5 in java.library.path: [/Users/andrej/Library/Java/Extensions, /Library/Java/Extensions, /Network/Library/Java/Extensions, /System/Library/Java/Extensions, /usr/lib/java, .]
	no swt-cocoa in java.library.path: [/Users/andrej/Library/Java/Extensions, /Library/Java/Extensions, /Network/Library/Java/Extensions, /System/Library/Java/Extensions, /usr/lib/java, .]
	no swt in java.library.path: [/Users/andrej/Library/Java/Extensions, /Library/Java/Extensions, /Network/Library/Java/Extensions, /System/Library/Java/Extensions, /usr/lib/java, .]
	Can't load library: /Users/andrej/.swt/lib/macosx/aarch64/libswt-cocoa-4948r5.jnilib
	Can't load library: /Users/andrej/.swt/lib/macosx/aarch64/libswt-cocoa.jnilib
	Can't load library: /Users/andrej/.swt/lib/macosx/aarch64/libswt.jnilib
	Can't load library: /Users/andrej/.swt/lib/macosx/aarch64/libswt-cocoa-4948r5.jnilib

	at org.eclipse.swt.internal.Library.loadLibrary(Library.java:348)
	at org.eclipse.swt.internal.Library.loadLibrary(Library.java:257)
	at org.eclipse.swt.internal.C.<clinit>(C.java:19)
	at org.eclipse.swt.widgets.Display.<clinit>(Display.java:107)
	at com.ibm.datapower.er.erGUI.<init>(Unknown Source)
	at com.ibm.datapower.er.ERTool.GetGUIArgs(Unknown Source)
	at com.ibm.datapower.er.ERTool.<init>(Unknown Source)
	at com.ibm.datapower.er.ERTool.main(Unknown Source)

Sun Java 1.8 library formulas may fail

I have noticed in Sun Java 1.8 some formulas are failing (no results will be found for that formula, other times it will work successfully, but rarely). This likely has to do with the multicore support, but haven't yet been able to pinpoint why this change in behavior for Sun Java 1.8.

I have not yet tested IBM Java 1.8, will also see if the same behavior is exhibited.

IBM Java 1.6 - OK
IBM Java 1.7 - OK
IBM Java 1.8 - OK
Sun Java 1.8 - Fails

Pull Files feature is too slow to extract all files in an archive

Need to review what is so slow reading from archive/writing to file. Want to speed it up to be more like a normal extraction utility if at all possible.

This is really detrimental to API Management archives, not as much an impact to DataPower.

mention the end of life of physical model as per technote

We should start to include the following hardware end of life in the error reports
http://www-01.ibm.com/support/docview.wss?uid=swg21317845

log4j libraries are affected by a remote code execution vulnerability

The log4j libraries in /framework/lib/ are outdated and and should be updated as they cause vulnerability scan issues (CVE-2021-44832).

• log4j-1.2.14.jar
• log4j-api-2.16.0.jar
• log4j-core-2.16.0.jar

double quote in formula for multiple gateways

just a reminder to push the new formla