newrelic-experimental / newrelic-java-mule-4

Instrumentation of the New Relic Java Agent for the Mule framework v4.x

New Relic Java Instrumentation for Mule 4.x

Instrumentation for the Mule 4.x framework.

Accouncement

The original releases of the Mule 4.x instrumentation used the New Relic Java Agent Asynchronous framework to tie the transaction together. But in some Mule applications it was resulting in some of the asynchronous tokens not being expired and some async segments not being ended. Because these items are stored in static caches it could consume large amounts of memory and cause problems. In Version 3.0 we started using distributed tracing headers to tie things together. This results in more transactions being reported but everything gets tied together in the Distributed Tracing view and it does not cause memory issues. If the desire is to have one conhesive transaction then use Release V2.0 provided that memory is not being consumed. If the distributed tracing view is acceptable then use Release V3.0

Installation

To install:

1. Download the latest release jar files.
2. In the New Relic Java directory (the one containing newrelic.jar), create a directory named extensions if it does not already exist.
3. Copy the downloaded jars into the extensions directory.
4. Restart the application.

Getting Started

Once installed, the instrumentation will track transactions through the various Mule components.

Building

Note that the jar necessary for building the Mule-Agent extension are no longer available in Maven. As a result, the extension has been removed from the build process. It can be built if you have access to either AnypointStudio or a Mule Server. Follow the instructions here to build: https://github.com/newrelic-experimental/newrelic-java-mule-4/blob/main/Mule-Agent-Build.md

Note that the a jar necessary for building the Mule-Batch-ee extension are no longer available in Maven. As a result, the extension has been removed from the build process. It can be built if you have access to either AnypointStudio or a Mule Server. Follow the instructions here to build: https://github.com/newrelic-experimental/newrelic-java-mule-4/blob/main/Mule-Batch-Build.md

Building the extension requires that Gradle is installed. To build the extension jars from source, follow these steps:

Build single extension

To build a single extension with name extension, do the following:

1. Set an environment variable NEW_RELIC_EXTENSIONS_DIR and set its value to the directory where you want the jar file built.
2. Run the command: ./gradlew extension:clean extension:install

Build all extensions

To build all extensions, do the following:

1. Set an environment variable NEW_RELIC_EXTENSIONS_DIR and set its value to the directory where you want the jar file built.
2. Run the command: ./gradlew clean install

Support

New Relic has open-sourced this project. This project is provided AS-IS WITHOUT WARRANTY OR DEDICATED SUPPORT. Issues and contributions should be reported to the project here on GitHub.

We encourage you to bring your experiences and questions to the Explorers Hub where our community members collaborate on solutions and new ideas.

Contributing

We encourage your contributions to improve [Project Name]! Keep in mind when you submit your pull request, you'll need to sign the CLA via the click-through using CLA-Assistant. You only have to sign the CLA one time per project. If you have any questions, or to execute our corporate CLA, required if your contribution is on behalf of a company, please drop us an email at [email protected].

A note about vulnerabilities

As noted in our security policy, New Relic is committed to the privacy and security of our customers and their data. We believe that providing coordinated disclosure by security researchers and engaging with the security community are important means to achieve our security goals.

If you believe you have found a security vulnerability in this project or any of New Relic's products or websites, we welcome and greatly appreciate you reporting it to New Relic through HackerOne.

License

[Project Name] is licensed under the Apache 2.0 License.

[If applicable: [Project Name] also uses source code from third-party libraries. You can find full details on which libraries are used and the terms under which they are licensed in the third-party notices document.]

newrelic-java-mule-4's Issues

Variables are printing 'null' with New Relic 8.9.0 enabled on Mulesoft Runtime 4.5.x & 4.6.x version

New Relic Support Case #00187431

Description

After upgrading to Mule 4.5.x & 4.6.x Runtime version and adding New Relic to the Wrapper.conf file, we encountered an issue where our application variables were becoming null.
Upon contacting Mule support, we were advised to comment out the New Relic Jar, after which the application variables started working normally again.
It appears that there is some interaction between New Relic and MuleSoft Runtime which is causing instability in our applications.

It New Relic Enabled works fine with 4.4.x Mulesoft Versions

Steps to Reproduce

Example of NULL

{"customerNumber":vars.lw9399res.customerNumber default 0,
"divisionNumber":vars.lw9399res.divisionNumber default 0,
"responseCode":payload.responseCode,
"productListCount":vars.lwreq.productListCount default 0,
"consumerId":vars.lwPriceReqHeader.consumerId,
"subConsumerId":vars.lwPriceReqHeader.subConsumerId,
"roundedProductCount":(ceil((vars.lwReq.productListCount default 0)/'${roundedProductCount}')) * 50,
"allVariables": vars,
"lw9399res": vars.lw9399res

}

Response:
{
"customerNumber": 0,
"divisionNumber": 0,
"responseCode": 9398,
"productListCount": 0,
"consumerId": null,
"subConsumerId": null,
"roundedProductCount": 0,
"allVariables": {
"exceptionFlag": "true"
},
"lw9399res": null
}

Expected Behavior

Example when New Relic is DISABLED on 4.5.x & 4.6.x

{
"customerNumber": 14374680,
"divisionNumber": 1933,
"responseCode": null,
"productListCount": 1,
"consumerId": "mule-test",
"subConsumerId": "mule-test",
"roundedProductCount": 50,
"allVariables": {
"lw9399res": {
"divisionNumber": 1933,
"customerNumber": 14374680,
"priceSourceFlag": "Y",
"priceRecommendationFlag": "Y",
"gaurdrailsFlag": "Y",
"costBasisDetailsFlag": "Y",
"lastWeekPriceFlag": "Y",
"pricingDateTime": "2024-02-15T15:21:00",
"lastWeekPriceDateTime": "2023-12-18T07:03:00",
"productListCount": 1,
"productList": [
{
"productNumber": 9449588,
"sequenceNumber": 1
}
]
},
"lwPriceReqHeader": {
"consumerId": "mule-test",
"subConsumerId": "mule-test",
"transactionId": "Mule-fdc45378-dfb1-448f-ae39-b48af3eebf42",
"correlationId": "1234444",
"timeZone": "America/Chicago"
},
"reqCheck": {
"levsLWFlag": "Y",
"lwDuration": "N",
"levsFlag": "N",
"reqDuration": "N"
},
"outboundHeaders": {
},
"lwReq": {
"divisionNumber": 1933,
"customerNumber": 14374680,
"priceSourceFlag": "Y",
"priceRecommendationFlag": "Y",
"gaurdrailsFlag": "Y",
"costBasisDetailsFlag": "Y",
"lastWeekPriceFlag": "Y",
"pricingDateTime": "2024-02-15T15:21:00",
"lastWeekPriceDateTime": "2023-12-18T07:03:00",
"productListCount": 1,
"productList": [
{
"productNumber": 9449588,
"sequenceNumber": 1
}
]
},
"apiStartTime": "2024-02-16T11:09:36.129-06:00"
},
"lw9399res": {
"divisionNumber": 1933,
"customerNumber": 14374680,
"priceSourceFlag": "Y",
"priceRecommendationFlag": "Y",
"gaurdrailsFlag": "Y",
"costBasisDetailsFlag": "Y",
"lastWeekPriceFlag": "Y",
"pricingDateTime": "2024-02-15T15:21:00",
"lastWeekPriceDateTime": "2023-12-18T07:03:00",
"productListCount": 1,
"productList": [
{
"productNumber": 9449588,
"sequenceNumber": 1
}
]
}
}

Relevant Logs / Console output

Your Environment

ALL ENVIRONMENTS

• ex: Browser name and version:
• ex: Operating System and version:

Additional context

Logs from newrelic_agent.log

2024-02-15T15:09:35,242-0600 [30379 133] com.newrelic INFO: Using default collector host: collector.newrelic.com
2024-02-15T15:09:35,242-0600 [30379 133] com.newrelic INFO: Using default metric ingest URI: https://metric-api.newrelic.com/metric/v1
2024-02-15T15:09:35,242-0600 [30379 133] com.newrelic INFO: Using default event ingest URI: https://insights-collector.newrelic.com/v1/accounts/events
2024-02-15T15:09:35,244-0600 [30379 133] com.newrelic INFO: Using default collector host: collector.newrelic.com
2024-02-15T15:09:35,244-0600 [30379 133] com.newrelic INFO: Using default metric ingest URI: https://metric-api.newrelic.com/metric/v1
2024-02-15T15:09:35,244-0600 [30379 133] com.newrelic INFO: Using default event ingest URI: https://insights-collector.newrelic.com/v1/accounts/events
2024-02-15T15:12:37,477-0600 [30379 193] com.newrelic INFO: Setting audit_mode to false
2024-02-15T15:12:37,478-0600 [30379 34] com.newrelic INFO: Host name is 1plsrmulas08.main.usfood.com, display host 1plsrmulas08.main.usfood.com for application null-sample
2024-02-15T15:12:37,478-0600 [30379 34] com.newrelic INFO: New Relic JFR Monitor is disabled: JFR config has not been enabled in the Java agent.
2024-02-15T15:12:38,077-0600 [30379 34] com.newrelic INFO: Collector redirection to collector.newrelic.com:443
2024-02-15T15:12:38,353-0600 [30379 34] com.newrelic INFO: Max payload size is 1,000,000 bytes
2024-02-15T15:12:38,353-0600 [30379 34] com.newrelic INFO: Agent run id: BclUHQHn2d5KAAb_qSEaP_Zlzn6YAAIBAAAnIQEAAHarAgQhGiq5AwAFOC45LjAAHDFwbHNybXVsYXMwOC5tYWluLnVzZm9vZC5jb20AC251bGwtc2FtcGxl
2024-02-15T15:12:38,353-0600 [30379 34] com.newrelic INFO: Agent [email protected]/null-sample connected to collector.newrelic.com:443
2024-02-15T15:12:38,353-0600 [30379 34] com.newrelic INFO: Reporting to: https://rpm.newrelic.com/accounts/458665/applications/555363001
2024-02-15T15:12:38,354-0600 [30379 34] com.newrelic INFO: Using default collector host: collector.newrelic.com
2024-02-15T15:12:38,354-0600 [30379 34] com.newrelic INFO: Using default metric ingest URI: https://metric-api.newrelic.com/metric/v1
2024-02-15T15:12:38,354-0600 [30379 34] com.newrelic INFO: Using default event ingest URI: https://insights-collector.newrelic.com/v1/accounts/events
2024-02-15T15:12:38,355-0600 [30379 34] com.newrelic INFO: Using default collector host: collector.newrelic.com
2024-02-15T15:12:38,355-0600 [30379 34] com.newrelic INFO: Using default metric ingest URI: https://metric-api.newrelic.com/metric/v1
2024-02-15T15:12:38,355-0600 [30379 34] com.newrelic INFO: Using default event ingest URI: https://insights-collector.newrelic.com/v1/accounts/events
2024-02-15T15:12:38,357-0600 [30379 34] com.newrelic INFO: Real user monitoring is enabled for application null-sample. Auto instrumentation is enabled.
2024-02-15T15:15:35,347-0600 [30379 194] com.newrelic INFO: Remote get_agent_commands call failed : javax.net.ssl.SSLException: Connection reset.
2024-02-15T15:16:08,371-0600 [30379 194] com.newrelic INFO: Remote get_agent_commands call failed : javax.net.ssl.SSLException: Connection reset.
2024-02-15T15:20:08,370-0600 [30379 194] com.newrelic INFO: Remote get_agent_commands call failed : javax.net.ssl.SSLException: Connection reset.
2024-02-15T15:29:02,150-0600 [30379 195] com.newrelic INFO: Remote span_event_data call failed : javax.net.ssl.SSLException: Connection reset.
2024-02-15T15:29:35,348-0600 [30379 194] com.newrelic INFO: Remote get_agent_commands call failed : javax.net.ssl.SSLException: Connection reset.
2024-02-15T15:44:32,150-0600 [30379 194] com.newrelic INFO: Remote get_agent_commands call failed : javax.net.ssl.SSLException: Connection reset.
2024-02-15T15:44:57,282-0600 [30379 195] com.newrelic INFO: Remote analytic_event_data call failed : javax.net.ssl.SSLException: Connection reset.
2024-02-15T15:45:52,281-0600 [30379 195] com.newrelic INFO: Remote analytic_event_data call failed : javax.net.ssl.SSLException: Connection reset.
2024-02-15T15:48:32,368-0600 [30379 194] com.newrelic INFO: Remote metric_data call failed : com.newrelic.agent.ForceRestartException: .
2024-02-15T15:48:32,368-0600 [30379 194] com.newrelic WARN: Received a ForceRestartException: com.newrelic.agent.ForceRestartException: . The agent will attempt to reconnect for data reporting. If this message continues, please contact support via https://support.newrelic.com/.
2024-02-15T15:48:32,643-0600 [30379 194] com.newrelic INFO: Host name is 1plsrmulas08.main.usfood.com, display host 1plsrmulas08.main.usfood.com for application mule-sit
2024-02-15T15:48:32,644-0600 [30379 194] com.newrelic INFO: New Relic JFR Monitor is disabled: JFR config has not been enabled in the Java agent.
2024-02-15T15:48:33,224-0600 [30379 194] com.newrelic INFO: Collector redirection to collector.newrelic.com:443
2024-02-15T15:48:33,458-0600 [30379 194] com.newrelic INFO: Max payload size is 1,000,000 bytes
2024-02-15T15:48:33,458-0600 [30379 194] com.newrelic INFO: Agent run id: BXaRTJnZrSuEAAb_qSDgeGtlzocIAAIBAAAnIQEAAHarAgQg4BXpAwAFOC45LjAAHDFwbHNybXVsYXMwOC5tYWluLnVzZm9vZC5jb20ACG11bGUtc2l0
2024-02-15T15:48:33,458-0600 [30379 194] com.newrelic INFO: Agent [email protected]/mule-sit connected to collector.newrelic.com:443
2024-02-15T15:48:33,458-0600 [30379 194] com.newrelic INFO: Reporting to: https://rpm.newrelic.com/accounts/458665/applications/551556585
2024-02-15T15:48:33,459-0600 [30379 194] com.newrelic INFO: Using default collector host: collector.newrelic.com
2024-02-15T15:48:33,459-0600 [30379 194] com.newrelic INFO: Using default metric ingest URI: https://metric-api.newrelic.com/metric/v1
2024-02-15T15:48:33,459-0600 [30379 194] com.newrelic INFO: Using default event ingest URI: https://insights-collector.newrelic.com/v1/accounts/events
2024-02-15T15:48:33,461-0600 [30379 194] com.newrelic INFO: Using default collector host: collector.newrelic.com
2024-02-15T15:48:33,461-0600 [30379 194] com.newrelic INFO: Using default metric ingest URI: https://metric-api.newrelic.com/metric/v1
2024-02-15T15:48:33,461-0600 [30379 194] com.newrelic INFO: Using default event ingest URI: https://insights-collector.newrelic.com/v1/accounts/events
2024-02-15T15:48:33,464-0600 [30379 194] com.newrelic INFO: Real user monitoring is enabled for application mule-sit. Auto instrumentation is enabled.
2024-02-15T16:01:28,479-0600 [30379 195] com.newrelic INFO: Remote analytic_event_data call failed : javax.net.ssl.SSLException: Connection reset.

Issue building with Gradle

I got another issue when building again, the issue with com.mulesoft.agent:mule-agent-web-interface:2.1.7 was passed

root@b72401fd7abf:/home/gradle# ./gradlew clean install

Task :Mule-Batch-ee:compileJava FAILED

FAILURE: Build failed with an exception.

• What went wrong:
  Execution failed for task ':Mule-Batch-ee:compileJava'.

Could not resolve all files for configuration ':Mule-Batch-ee:compileClasspath'.
Could not find com.mulesoft.licm:licm:2.1.4.
Searched in the following locations:
- file:/root/.m2/repository/com/mulesoft/licm/licm/2.1.4/licm-2.1.4.pom
- https://repo.maven.apache.org/maven2/com/mulesoft/licm/licm/2.1.4/licm-2.1.4.pom
- https://repository.mulesoft.org/releases/com/mulesoft/licm/licm/2.1.4/licm-2.1.4.pom
- https://repository.mulesoft.org/nexus/content/repositories/public/com/mulesoft/licm/licm/2.1.4/licm-2.1.4.pom
- https://repo1.maven.org/maven2/com/mulesoft/licm/licm/2.1.4/licm-2.1.4.pom
Required by:
project :Mule-Batch-ee > com.mulesoft.mule.runtime.modules:mule-module-batch-ee:4.2.1 > com.mulesoft.mule.runtime:mule-core-ee:4.2.1

• Try:
  Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

• Get more help at https://help.gradle.org

BUILD FAILED in 8s
44 actionable tasks: 24 executed, 20 up-to-date

Error on Gradle build

I got this error when building extensions

Task :checkForDependencies
Starting a Gradle Daemon, 1 busy Daemon could not be reused, use --status for details

Task :getAgent
Unable to get progress logger. Download progress will not be displayed.

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.4.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 47s
1 actionable task: 1 executed

Task :extractJars

BUILD SUCCESSFUL in 1m 35s
1 actionable task: 1 executed

Task :cleanUp

BUILD SUCCESSFUL in 20s
1 actionable task: 1 executed

Task :Mule-Agent:compileJava FAILED

FAILURE: Build failed with an exception.

• What went wrong:
  Execution failed for task ':Mule-Agent:compileJava'.

Could not resolve all files for configuration ':Mule-Agent:compileClasspath'.
Could not find com.mulesoft.agent:mule-agent-web-interface:2.1.7.
Searched in the following locations:
- file:/root/.m2/repository/com/mulesoft/agent/mule-agent-web-interface/2.1.7/mule-agent-web-interface-2.1.7.pom
- https://repo.maven.apache.org/maven2/com/mulesoft/agent/mule-agent-web-interface/2.1.7/mule-agent-web-interface-2.1.7.pom
- https://repository.mulesoft.org/releases/com/mulesoft/agent/mule-agent-web-interface/2.1.7/mule-agent-web-interface-2.1.7.pom
- https://repository.mulesoft.org/nexus/content/repositories/public/com/mulesoft/agent/mule-agent-web-interface/2.1.7/mule-agent-web-interface-2.1.7.pom
- https://repo1.maven.org/maven2/com/mulesoft/agent/mule-agent-web-interface/2.1.7/mule-agent-web-interface-2.1.7.pom
Required by:
project :Mule-Agent

• Try:
  Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

• Get more help at https://help.gradle.org

BUILD FAILED in 2m 50s
29 actionable tasks: 4 executed, 25 up-to-date

Also is there anywhere to download the extenions.tar file?, I saw on the documentations there was extentions.tar but can not found it.

[Repolinter] Open Source Policy Issues

Repolinter Report

🤖This issue was automatically generated by repolinter-action, developed by the Open Source and Developer Advocacy team at New Relic. This issue will be automatically updated or closed when changes are pushed. If you have any problems with this tool, please feel free to open a GitHub issue or give us a ping in #help-opensource.

This Repolinter run generated the following results:

❗ Error	❌ Fail	⚠️ Warn	✅ Pass	Ignored	Total
0	0	1	5	0	6

• Warning
  • ⚠️ third-party-notices-file-exists
• Passed
  • ✅ license-file-exists
  • ✅ readme-file-exists
  • ✅ readme-starts-with-experimental-header
  • ✅ readme-contains-link-to-security-policy
  • ✅ code-of-conduct-should-not-exist-here

Warning #

Click to see rules

⚠️ third-party-notices-file-exists #

A THIRD_PARTY_NOTICES.md file can be present in your repository to grant attribution to all dependencies being used by this project. This document is necessary if you are using third-party source code in your project, with the exception of code referenced outside the project's compiled/bundled binary (ex. some Java projects require modules to be pre-installed in the classpath, outside the project binary and therefore outside the scope of the THIRD_PARTY_NOTICES). Please review your project's dependencies and create a THIRD_PARTY_NOTICES.md file if necessary. For JavaScript projects, you can generate this file using the oss-cli. For more information please visit https://docs.google.com/document/d/1y644Pwi82kasNP5VPVjDV8rsmkBKclQVHFkz8pwRUtE/view. Did not find a file matching the specified patterns. Below is a list of files or patterns that failed:

• THIRD_PARTY_NOTICES*
• THIRD-PARTY-NOTICES*
• THIRDPARTYNOTICES*

Passed #

Click to see rules

✅ license-file-exists #

Found file (LICENSE). New Relic requires that all open source projects have an associated license contained within the project. This license must be permissive (e.g. non-viral or copyleft), and we recommend Apache 2.0 for most use cases. For more information please visit https://docs.google.com/document/d/1vML4aY_czsY0URu2yiP3xLAKYufNrKsc7o4kjuegpDw/edit.

✅ readme-file-exists #

Found file (README.md). New Relic requires a README file in all projects. This README should give a general overview of the project, and should point to additional resources (security, contributing, etc.) where developers and users can learn further. For more information please visit https://github.com/newrelic/open-source-tools/tree/master/nerdpacks/oss-template.

✅ readme-starts-with-experimental-header #

The first 5 lines contain all of the requested patterns. (README.md). The README of an experimental project should have an experimental header at the start of the README. If you already have an experimental header and this rule is failing, your header may be out of date, and you should update your header with the suggested one below. For more information please visit https://opensource.newrelic.com/oss-category/.

✅ readme-contains-link-to-security-policy #

Contains a link to the security policy for this repository (README.md). New Relic recommends putting a link to the open source security policy for your project (https://github.com/newrelic-experimental/<repo-name>/security/policy or ../../security/policy) in the README. For an example of this, please see the "a note about vulnerabilities" section of the Open By Default repository. For more information please visit https://nerdlife.datanerd.us/new-relic/security-guidelines-for-publishing-source-code.

✅ code-of-conduct-should-not-exist-here #

New Relic has moved the CODE_OF_CONDUCT file to a centralized location where it is referenced automatically by every repository in the New Relic organization. Because of this change, any other CODE_OF_CONDUCT file in a repository is now redundant and should be removed. Note that you will need to adjust any links to the local CODE_OF_CONDUCT file in your documentation to point to the central file (README and CONTRIBUTING will probably have links that need updating). For more information please visit https://docs.google.com/document/d/1y644Pwi82kasNP5VPVjDV8rsmkBKclQVHFkz8pwRUtE/view. Did not find a file matching the specified patterns. All files passed this test.