newrelic / newrelic-java-kotlin-coroutines

Provides instrumentation for Kotlin Coroutines

New Relic Java Instrumentation for Kotlin Coroutines

Provides instrumentation for Kotlin Coroutines. In particular it will trace the coroutine from its start, suspend and resume. It does this across multilple threads.

Supported Versions

Kotlin-Coroutines-1.0 - all 1.0.x versions. Kotlin-Coroutines-1.1 - all 1.1.x versions.
Kotlin-Coroutines-1.2 - all 1.2.x versions.
Kotlin-Coroutines-1.3 - all 1.3.x versions.
Kotlin-Coroutines-1.4 - all 1.4.x and later versions.

Installation

To use this instrumentation.
Download the latest release.
In the New Relic Java Agent directory (directory containing newrelic.jar), create a directory named extensions if it does not already exist.
Copy the jars into the extensions directory.
Restart the application.

Getting Started

After deployment of the instrumentation jars, you should be able to see the invocation of a coroutine from start to finish across any threads that it executes on.

Usage

Instrumentation of methods with high invocation rates can lead to CPU overhead especially if its average response time is very small (i.e. less than a few milliseconds). Therefore it is possible to configure the agent to ignore certain suspend methods, dispatched tasks and continuation resumeWiths. This configuation is done in the newrelic.yml file.

Finding Possible Methods to Ignore

Run the following NRQL query where appName is the name of the application using Kotlin Coroutines.

SELECT rate(count(newrelic.timeslice.value), 1 MINUTE) FROM Metric WHERE (metricTimesliceName Like 'Custom/DispatchedTask/%' or metricTimesliceName Like 'Custom/WrappedSuspend/%' or metricTimesliceName Like 'Custom/ContinuationWrapper%') AND appName='appName' SINCE 24 HOURS AGO FACET metricTimesliceName

The following is a screenshot of DispatchedTasks

At minumum consider ignoring anything over 50K.

Below each rate is the name of the metric, it has the form Custom/DispatchedTask/..., or Custom/WrappedSuspend/... or Custom/ContinuationWrapper/... depending on the query that was run. Collect a list of the remaining metric name (i.e. the ...).

Configuring Methods to Ignore

To configure methods to ignore, edit the newrelic.yml file in the New Relic Java Agent directory.

1. Find the following lines in newrelic.yml

2. Insert the following lines BEFORE the above lines being mindful of spaces at the beginning of each line (2 on first, 4 on second, 6 on third). Each list is comma separated listed from ones found in the previous section. If none are none for the particular line it can be omitted.

3. Save newrelic.yml

Note that these setting are dynamic, so typically the agent should pick up changes within a minute or so and implement the changes without having to restart.

Building

If you make changes to the instrumentation code and need to build the instrumentation jars, follow these steps

1. Set environment variable NEW_RELIC_EXTENSIONS_DIR. Its value should be the directory where you want to build the jars (i.e. the extensions directory of the Java Agent).
2. Build one or all of the jars.
   a. To build one jar, run the command: gradlew moduleName:clean moduleName:install
   b. To build all jars, run the command: gradlew clean install
3. Restart the application

Support

New Relic has open-sourced this project. Issues and contributions should be reported to the project here on GitHub.

We encourage you to bring your experiences and questions to the Explorers Hub where our community members collaborate on solutions and new ideas.

Contributing

We encourage your contributions to improve Salesforce Commerce Cloud for New Relic Browser! Keep in mind when you submit your pull request, you'll need to sign the CLA via the click-through using CLA-Assistant. You only have to sign the CLA one time per project. If you have any questions, or to execute our corporate CLA, required if your contribution is on behalf of a company, please drop us an email at [email protected].

A note about vulnerabilities

As noted in our security policy, New Relic is committed to the privacy and security of our customers and their data. We believe that providing coordinated disclosure by security researchers and engaging with the security community are important means to achieve our security goals.

If you believe you have found a security vulnerability in this project or any of New Relic's products or websites, we welcome and greatly appreciate you reporting it to New Relic through HackerOne.

License

Kotlin Coroutine Instrumentation is licensed under the Apache 2.0 License.

newrelic-java-kotlin-coroutines's Issues

ignores.suspend not filtering UndispatchedCoroutine

I am not absolutely this should work, but it seems I can't ignore suspend traces for example for Custom/WrappedSuspend/UndispatchedCoroutine .

I have added the following to newrelic.yml:

  Coroutines:
    ignores:
      continuations: jetty-call-handler
      suspends: UndispatchedCoroutine,jetty-call-handler
      dispatched: io.ktor.utils.io.jvm.javaio.OutputAdapter$loop$1$loop$1,io.ktor.utils.io.ByteBufferChannel$writeSuspend$1

The filtering on dispatched works, but for suspends it does not seem to work. Am I on the right track here?

Publish ".tgz" along with ".jar" files for release

Summary

Releases of this extension are currently provided as jar file artifacts. The New Relic buildpack only supports adding java agent extensions (such as this) that are in gzipped tar format. It would be convenient to be able to specify the URL to this extension packaged as a ".tgz" artifact.

Desired Behavior

It would be desirable to be able to specify the value of BP_NEW_RELIC_EXT_URI for the New Relic Buildpack to be a Gzipped tar release artifact of this repository.

Possible Solution

Zip each jar file artifact and add them to the release artifacts.

Additional context

Will Coroutines 1.6 work with this library

Summary

It looks like 1.6 support hasn't been mentioned, it may just mean a doco update but my initial integration doesn't seem to work, so I'm wondering if there's work that needs to be done.

I thought it worth asking here if there's any issue while I troubleshoot!

Unexpected exception thrown in class transformer

Description

Sometime we've got a logged exception when new relic agent debug logging enabled in the production

Steps to Reproduce

There is no clean way to reproduce it.

Relevant Logs / Console output

com.newrelic FINE: Unexpected exception thrown in class transformer: 
jdk.internal.loader.ClassLoaders$AppClassLoader@9e89d68
--kotlinx/coroutines/CancellableContinuationImpl
java.lang.NullPointerException: null

Your Environment

Kotlin 1.5.20
Kotlin-Coroutines: 1.5.0
Ktor 1.6.3
NewRelic Agent: 7.3.0
Newrelic Coroutines extension: 1.3/Kotlin-Coroutines_1.4

Transaction linking in co-routines when exceptions are thrown

Hi,

I am no expert with co-routines, so I am not sure if this should work or not, but to my limited understanding I think it should.

We are using KTor (which use co-routines heavily), and this extension made our world a lot better. Thanks. :) But it seems we are getting some weird behavior in cases where we throw exceptions.

Because the original routine suspended (see the code below), the catch of the exception (sometimes) resumes in a different thread. Then we get Transaction=Unknown for those cases. To my understanding this extension should handle co-routines jumping threads, but not really sure it is possible in the case of Exceptions? Using tokens to link in (in the catch clause), of course works.

The parallel code that is triggering this behavior is this:

suspend fun <A, B> Iterable<A>.parallelMap(concurrency: Int = 10, f: suspend (A) -> B): List<B> {
    return coroutineScope {
        val semaphore = Semaphore(concurrency)
        map {
            async {
                semaphore.withPermit {
                    f(it)
                }
            }
        }.awaitAll()
    }
}

Exchanging a call to this function with a simple .map {} makes the weird behavior disappear.

So my question is really can/should this be handled? Right now it is throwing some of our error tracking and stats off. :)

Thanks,
Anders,

[Repolinter] Open Source Policy Issues

Repolinter Report

🤖This issue was automatically generated by repolinter-action, developed by the Open Source and Developer Advocacy team at New Relic. This issue will be automatically updated or closed when changes are pushed. If you have any problems with this tool, please feel free to open a GitHub issue or give us a ping in #help-opensource.

This Repolinter run generated the following results:

❗ Error	❌ Fail	⚠️ Warn	✅ Pass	Ignored	Total
0	1	1	5	0	7

• Fail
  • ❌ readme-contains-forum-topic
• Warning
  • ⚠️ third-party-notices-file-exists
• Passed
  • ✅ license-file-exists
  • ✅ readme-file-exists
  • ✅ readme-starts-with-community-header
  • ✅ readme-contains-link-to-security-policy
  • ✅ code-of-conduct-should-not-exist-here

Fail #

❌ readme-contains-forum-topic #

Doesn't contain a link to the appropriate forum.newrelic.com topic (README.md). New Relic recommends directly linking the your appropriate forum.newrelic.com topic in the README, allowing developer an alternate method of getting support. For more information please visit https://nerdlife.datanerd.us/new-relic/security-guidelines-for-publishing-source-code.

Warning #

Click to see rules

⚠️ third-party-notices-file-exists #

A THIRD_PARTY_NOTICES.md file can be present in your repository to grant attribution to all dependencies being used by this project. This document is necessary if you are using third-party source code in your project, with the exception of code referenced outside the project's compiled/bundled binary (ex. some Java projects require modules to be pre-installed in the classpath, outside the project binary and therefore outside the scope of the THIRD_PARTY_NOTICES). Please review your project's dependencies and create a THIRD_PARTY_NOTICES.md file if necessary. For JavaScript projects, you can generate this file using the oss-cli. For more information please visit https://docs.google.com/document/d/1y644Pwi82kasNP5VPVjDV8rsmkBKclQVHFkz8pwRUtE/view. Did not find a file matching the specified patterns. Below is a list of files or patterns that failed:

• THIRD_PARTY_NOTICES*
• THIRD-PARTY-NOTICES*
• THIRDPARTYNOTICES*

Passed #

Click to see rules

✅ license-file-exists #

Found file (LICENSE). New Relic requires that all open source projects have an associated license contained within the project. This license must be permissive (e.g. non-viral or copyleft), and we recommend Apache 2.0 for most use cases. For more information please visit https://docs.google.com/document/d/1vML4aY_czsY0URu2yiP3xLAKYufNrKsc7o4kjuegpDw/edit.

✅ readme-file-exists #

Found file (README.md). New Relic requires a README file in all projects. This README should give a general overview of the project, and should point to additional resources (security, contributing, etc.) where developers and users can learn further. For more information please visit https://github.com/newrelic/open-by-default.

✅ readme-starts-with-community-header #

The first 5 lines contain all of the requested patterns. (README.md). The README of a community project should have a community project header at the start of the README. If you already have a community project header and this rule is failing, your header may be out of date, and you should update your header with the suggested one below. For more information please visit https://opensource.newrelic.com/oss-category/.

✅ readme-contains-link-to-security-policy #

Contains a link to the security policy for this repository (README.md). New Relic recommends putting a link to the open source security policy for your project (https://github.com/newrelic/<repo-name>/security/policy or ../../security/policy) in the README. For an example of this, please see the "a note about vulnerabilities" section of the Open By Default repository. For more information please visit https://nerdlife.datanerd.us/new-relic/security-guidelines-for-publishing-source-code.

✅ code-of-conduct-should-not-exist-here #

New Relic has moved the CODE_OF_CONDUCT file to a centralized location where it is referenced automatically by every repository in the New Relic organization. Because of this change, any other CODE_OF_CONDUCT file in a repository is now redundant and should be removed. Note that you will need to adjust any links to the local CODE_OF_CONDUCT file in your documentation to point to the central file (README and CONTRIBUTING will probably have links that need updating). For more information please visit https://docs.google.com/document/d/1y644Pwi82kasNP5VPVjDV8rsmkBKclQVHFkz8pwRUtE/view. Did not find a file matching the specified patterns. All files passed this test.

[Repolinter] Open Source Policy Issues

Repolinter Report

🤖This issue was automatically generated by repolinter-action, developed by the Open Source and Developer Advocacy team at New Relic. This issue will be automatically updated or closed when changes are pushed. If you have any problems with this tool, please feel free to open a GitHub issue or give us a ping in #help-opensource.

This Repolinter run generated the following results:

❗ Error	❌ Fail	⚠️ Warn	✅ Pass	Ignored	Total
0	0	1	6	0	7

• Warning
  • ⚠️ third-party-notices-file-exists
• Passed
  • ✅ license-file-exists
  • ✅ readme-file-exists
  • ✅ readme-starts-with-community-header
  • ✅ readme-contains-link-to-security-policy
  • ✅ readme-contains-discuss-topic
  • ✅ code-of-conduct-should-not-exist-here

Warning #

Click to see rules

⚠️ third-party-notices-file-exists #

A THIRD_PARTY_NOTICES.md file can be present in your repository to grant attribution to all dependencies being used by this project. This document is necessary if you are using third-party source code in your project, with the exception of code referenced outside the project's compiled/bundled binary (ex. some Java projects require modules to be pre-installed in the classpath, outside the project binary and therefore outside the scope of the THIRD_PARTY_NOTICES). Please review your project's dependencies and create a THIRD_PARTY_NOTICES.md file if necessary. For JavaScript projects, you can generate this file using the oss-cli. For more information please visit https://docs.google.com/document/d/1y644Pwi82kasNP5VPVjDV8rsmkBKclQVHFkz8pwRUtE/view. Did not find a file matching the specified patterns. Below is a list of files or patterns that failed:

• THIRD_PARTY_NOTICES*
• THIRD-PARTY-NOTICES*
• THIRDPARTYNOTICES*

Passed #

Click to see rules

✅ license-file-exists #

Found file (LICENSE). New Relic requires that all open source projects have an associated license contained within the project. This license must be permissive (e.g. non-viral or copyleft), and we recommend Apache 2.0 for most use cases. For more information please visit https://docs.google.com/document/d/1vML4aY_czsY0URu2yiP3xLAKYufNrKsc7o4kjuegpDw/edit.

✅ readme-file-exists #

Found file (README.md). New Relic requires a README file in all projects. This README should give a general overview of the project, and should point to additional resources (security, contributing, etc.) where developers and users can learn further. For more information please visit https://github.com/newrelic/open-by-default.

✅ readme-starts-with-community-header #

The first 5 lines contain all of the requested patterns. (README.md). The README of a community project should have a community project header at the start of the README. If you already have a community project header and this rule is failing, your header may be out of date, and you should update your header with the suggested one below. For more information please visit https://opensource.newrelic.com/oss-category/.

✅ readme-contains-link-to-security-policy #

Contains a link to the security policy for this repository (README.md). New Relic recommends putting a link to the open source security policy for your project (https://github.com/newrelic/<repo-name>/security/policy or ../../security/policy) in the README. For an example of this, please see the "a note about vulnerabilities" section of the Open By Default repository. For more information please visit https://nerdlife.datanerd.us/new-relic/security-guidelines-for-publishing-source-code.

✅ readme-contains-discuss-topic #

Contains a link to the appropriate discuss.newrelic.com topic (README.md). New Relic recommends directly linking the your appropriate discuss.newrelic.com topic in the README, allowing developer an alternate method of getting support. For more information please visit https://nerdlife.datanerd.us/new-relic/security-guidelines-for-publishing-source-code.

✅ code-of-conduct-should-not-exist-here #

New Relic has moved the CODE_OF_CONDUCT file to a centralized location where it is referenced automatically by every repository in the New Relic organization. Because of this change, any other CODE_OF_CONDUCT file in a repository is now redundant and should be removed. Note that you will need to adjust any links to the local CODE_OF_CONDUCT file in your documentation to point to the central file (README and CONTRIBUTING will probably have links that need updating). For more information please visit https://docs.google.com/document/d/1y644Pwi82kasNP5VPVjDV8rsmkBKclQVHFkz8pwRUtE/view. Did not find a file matching the specified patterns. All files passed this test.

[Repolinter] Open Source Policy Issues

Repolinter Report

🤖This issue was automatically generated by repolinter-action, developed by the Open Source and Developer Advocacy team at New Relic. This issue will be automatically updated or closed when changes are pushed. If you have any problems with this tool, please feel free to open a GitHub issue or give us a ping in #help-opensource.

This Repolinter run generated the following results:

❗ Error	❌ Fail	⚠️ Warn	✅ Pass	Ignored	Total
0	0	1	5	0	6

• Warning
  • ⚠️ third-party-notices-file-exists
• Passed
  • ✅ license-file-exists
  • ✅ readme-file-exists
  • ✅ readme-starts-with-experimental-header
  • ✅ readme-contains-link-to-security-policy
  • ✅ code-of-conduct-should-not-exist-here

Warning #

Click to see rules

⚠️ third-party-notices-file-exists #

A THIRD_PARTY_NOTICES.md file can be present in your repository to grant attribution to all dependencies being used by this project. This document is necessary if you are using third-party source code in your project, with the exception of code referenced outside the project's compiled/bundled binary (ex. some Java projects require modules to be pre-installed in the classpath, outside the project binary and therefore outside the scope of the THIRD_PARTY_NOTICES). Please review your project's dependencies and create a THIRD_PARTY_NOTICES.md file if necessary. For JavaScript projects, you can generate this file using the oss-cli. For more information please visit https://docs.google.com/document/d/1y644Pwi82kasNP5VPVjDV8rsmkBKclQVHFkz8pwRUtE/view. Did not find a file matching the specified patterns. Below is a list of files or patterns that failed:

• THIRD_PARTY_NOTICES*
• THIRD-PARTY-NOTICES*
• THIRDPARTYNOTICES*

Passed #

Click to see rules

✅ license-file-exists #

Found file (LICENSE). New Relic requires that all open source projects have an associated license contained within the project. This license must be permissive (e.g. non-viral or copyleft), and we recommend Apache 2.0 for most use cases. For more information please visit https://docs.google.com/document/d/1vML4aY_czsY0URu2yiP3xLAKYufNrKsc7o4kjuegpDw/edit.

✅ readme-file-exists #

Found file (README.md). New Relic requires a README file in all projects. This README should give a general overview of the project, and should point to additional resources (security, contributing, etc.) where developers and users can learn further. For more information please visit https://github.com/newrelic/open-source-tools/tree/master/nerdpacks/oss-template.

✅ readme-starts-with-experimental-header #

The first 5 lines contain all of the requested patterns. (README.md). The README of an experimental project should have an experimental header at the start of the README. If you already have an experimental header and this rule is failing, your header may be out of date, and you should update your header with the suggested one below. For more information please visit https://opensource.newrelic.com/oss-category/.

✅ readme-contains-link-to-security-policy #

Contains a link to the security policy for this repository (README.md). New Relic recommends putting a link to the open source security policy for your project (https://github.com/newrelic-experimental/<repo-name>/security/policy or ../../security/policy) in the README. For an example of this, please see the "a note about vulnerabilities" section of the Open By Default repository. For more information please visit https://nerdlife.datanerd.us/new-relic/security-guidelines-for-publishing-source-code.

✅ code-of-conduct-should-not-exist-here #

New Relic has moved the CODE_OF_CONDUCT file to a centralized location where it is referenced automatically by every repository in the New Relic organization. Because of this change, any other CODE_OF_CONDUCT file in a repository is now redundant and should be removed. Note that you will need to adjust any links to the local CODE_OF_CONDUCT file in your documentation to point to the central file (README and CONTRIBUTING will probably have links that need updating). For more information please visit https://docs.google.com/document/d/1y644Pwi82kasNP5VPVjDV8rsmkBKclQVHFkz8pwRUtE/view. Did not find a file matching the specified patterns. All files passed this test.