netflix / mantis Goto Github PK

Hello I would like to make the document of requirements for the project. Could you give me some information about the project?

Context

Object Mappers are used throughout the codebase. The problem with this is that they are all configured differently. Therefore, if we need to introduce a new module, we will have to change all the ObjectMappers.

Steps to reproduce

Explain the steps to reliably reproduce the issue.

Expected behavior

Explain what you think should happen.

Actual Behavior

Explain what actually happens instead.

Context

We would like to be on the latest version of gradle.

Context

There are some ordering assumptions that are made in the test io.mantisrx.master.jobcluster.JobClusterTest#testCronTriggersSLAToKillOld. This assumption may or may not hold true depending on timing and hence is flaky.

Steps to reproduce

Run the test a few times.

Expected behavior

The test should always succeed.

Actual Behavior

The test should fail sometimes.

Context

We've deployed Mantis and are able to get the master to accept work and submit it to an ASG backed DEFAULT ResourceClusterProvider. However the default doesn't come with autoscaling and we'd like to test our implementation of ResourceClusterProvider as well as have our cluster autoscale.

What request and payload do we need to submit to the master to assign an autoscaling policy to DEFAULT? The docs don't have anything here but I'd be happy to submit a pull request to them to get a section started on managing ResourceClusterProviders.

CVE-2019-20445 CVE-2019-20444 CVE-2019-9512 CVE-2019-16869 CVE-2019-9514
Recommended upgrade version：4.1.60.Final

Context

mantis-examples-mantis-publish-web-sample subproject under mantis-examples does not work with the dependency recommender plugin.

Steps to reproduce

Uncomment the relevant line in settings.gradle#L42-43; looks something like //include 'mantis-examples:mantis-examples-mantis-publish-web-sample'

Expected behavior

./gradlew generateLock saveLock runs successfully without issues.

Actual Behavior

./gradlew generateLock saveLock fails instead with the stack trace below.

Execution failed for task ':mantis-examples:mantis-examples-mantis-publish-web-sample:generateLock'.
> Could not resolve all dependencies for configuration ':mantis-examples:mantis-examples-mantis-publish-web-sample:grettyRunnerJetty7'.
   > Could not find org.gretty:gretty-runner-jetty7:2.2.0.
     Searched in the following locations:
       - https://repo.maven.apache.org/maven2/org/gretty/gretty-runner-jetty7/2.2.0/gretty-runner-jetty7-2.2.0.pom
       - https://netflixoss.jfrog.io/artifactory/maven-oss-candidates/org/gretty/gretty-runner-jetty7/2.2.0/gretty-runner-jetty7-2.2.0.pom
     Required by:
         project :mantis-examples:mantis-examples-mantis-publish-web-sample

* Try:

Context

We use the slf4j logger API to emit logs from within all the actors. All these actors have a unique way of identifying themselves. For instance, the JobActor represents an instance of a running Job with a unique JobID. Thus, it would be nice to ensure all the logs emitted from the JobActor contain this information. Today, this is done in an ad-hoc fashion. Some logs have the JobID, while others don't. It would be nice to develop a more structured way of solving this.

Goals

Look at the below line from JobActor.java to understand what the problem is.

LOGGER.info("Initializing Job {}", jobId);

We manually add the jobID in log line above. However, in other places, we fail to do this—for instance, look at the example below (also from JobActor.java).

LOGGER.info("Stored mantis job");

Instead, it would be nice to have a more structured way of solving this problem—something like the one below.

jobLogger.info("Whatever");

And the expectation is that jobLogger adds the jobID to every emitted logline by default.

Context

There is currently a bug in the SSE implementation when compression is enabled. Mantis compiles a batch of data delimiting each event with a fixed delimiter $$$. In the case where an event contains the string $$$ the event will be erroneously split in two and fail to deserialize downstream.

Steps to reproduce

Create a job producing events which contain $$$ and subscribe to it with compression enabled.

Expected behavior

The events should not be incorrectly bifurcated downstream.

Actual Behavior

The events are bifurcated when being decompressed downstream, when the list is split the legitimate contents of the string are misidentified as a delimiter and the event is erroneously bifurcated.

Context

Hello, I am new to the project and after following the README, the IntelliJ project setup was successful, the build is failing however, with ComparisonFailure, AssertionErrors and NoClassDefFoundErrors

Steps to reproduce

Running the ./gradlew clean build command produces the error
PFA screenshot of the error for reference

Expected behavior

Successful build

Actual Behavior

Build fails.

Context

The Official Website is down

Steps to reproduce

Navigate to https://netflix.github.io/mantis/

Expected behavior

I should see the docs website

Actual Behavior

I see

Context

Mantis has several sub-projects, and every sub-project depends on several 3rd party dependencies. Therefore, we would like consistency around how the 3rd party dependencies are defined and their versions. This would ensure a consistent set of jars in the classpath irrespective of the project that's getting deployed.

To achieve this goal, we introduced a new format for defining 3rd party dependencies in subprojects, as shown below.

implementation libraries.slf4jApi (correct)
implementation "org.slf4j:slf4j-api:${versions.slf4j}" (wrong)
implementation "org.slf4j:slf4j-api:1.2.3" (wrong)

Corresponding root.gradle changes:

ext.libraries = [
  slf4jApi       : "org.slf4j:slf4j-api:${versions.slf4j}"
]

This type of definition achieves the goal defined earlier. However, it is yet to be fully adopted. This ticket aims to do this cleanup for every subproject within mantis and create one or more PRs.

If you will work on this ticket, please create multiple PRs or have multiple commits for every subproject you have refactored to ensure it's easy to review the code.

There are compile warnings in the code due to:

• deprecated APIs
• missing @Nullable/@NotNull annotations
• incomplete javadocs
• lambda can be replaced with method references
• raw usage of generic types
• unnecessary type parameters (like Map<String> m = new HashMap<String>())
• anonymous class can be replaced with lambdas
• ... and more.

We don't need to solve all of them, but a good coding exercise is to address most those compile warnings when possible. Guidelines are:

1. Not change code logic, which should be addressed and filed PR separately
2. Not change public interfaces (class, methods and fields) that would break other modules or applications. But it's safe to rename private fields or methods when necessary.
3. Prefer readability to more concise code
4. Improve tests in the module is welcome but not required
5. Compile and pass all existing tests.

This tracks the effort for the mantis-server-worker module to make the code change scoped.

There are compile warnings in the code due to:

• deprecated APIs
• missing @Nullable/@NotNull annotations
• incomplete javadocs
• lambda can be replaced with method references
• raw usage of generic types
• unnecessary type parameters (like Map<String> m = new HashMap<String>())
• anonymous class can be replaced with lambdas
  and more.

We don't need to solve all of them, but a good coding exercise is to address most those compile warnings when possible. Guidelines are:

1. Not change code logic, which should be addressed and filed PR separately
2. Not change public interfaces (class, methods and fields) that would break other modules or applications. But it's safe to rename private fields or methods when necessary.
3. Prefer readability to more concise code
4. Improve tests in the module is welcome but not required
5. Compile and pass all existing tests.

This tracks the effort for the mantis-server-agent module to make the code change scoped.

There are compile warnings in the code due to:

• deprecated APIs
• missing @Nullable/@NotNull annotations
• incomplete javadocs
• lambda can be replaced with method references
• raw usage of generic types
• unnecessary type parameters (like Map<String> m = new HashMap<String>())
• anonymous class can be replaced with lambdas
• ... and more.

We don't need to solve all of them, but a good coding exercise is to address most those compile warnings when possible. Guidelines are:

1. Not change code logic, which should be addressed and filed PR separately
2. Not change public interfaces (class, methods and fields) that would break other modules or applications. But it's safe to rename private fields or methods when necessary.
3. Prefer readability to more concise code
4. Improve tests in the module is welcome but not required
5. Compile and pass all existing tests.

This tracks the effort for the mantis-control-plane-core module to make the code change scoped.

Context

We would like to have Mantis compiled using JDK 17. However, since mantis-runtime is used in several jobs built using JDK 17, we cannot target 17-level compatibility at runtime. Thus, we would like the overall codebase to be compiled using Java 17 while still targeting JRE8-level compatibility.

Context

Current naming on SchedulingInfo Builders is a mixed of (SingleWorkerStage vs add.*Stage).

Expected behavior

Use consistent naming convention 'add.*Stage'.

Context

This is similar to #384 but in the context of the JobClusterActor.

Expected Behavior

Every emitted logline from the JobClusterActor has the cluster identifier appended automatically.

Seems like the test is broken when running locally. Let's fix or improve it.

java.lang.AssertionError: Unexpected status code! expected:<200 OK> but was:<500 Internal Server Error>
  Request was:      HttpRequest(HttpMethod(POST),/api/v1/resourceClusters/myCluster/disableTaskExecutors,List(),HttpEntity.Strict(application/json,81 bytes total),HttpProtocol(HTTP/1.1))
  Route result was: FulfilledFuture(Complete(HttpResponse(500 Internal Server Error,List(Access-Control-Allow-Origin: *),HttpEntity.Strict(text/plain; charset=UTF-8,141 bytes total),HttpProtocol(HTTP/1.1))))

	at akka.http.javadsl.testkit.JUnitRouteTestBase$$anon$1.reportDetails(JUnitRouteTest.scala:48)
	at akka.http.javadsl.testkit.JUnitRouteTestBase$$anon$1.assertEquals(JUnitRouteTest.scala:31)
	at akka.http.javadsl.testkit.TestRouteResult.assertEqualsKind(TestRouteResult.scala:227)
	at akka.http.javadsl.testkit.TestRouteResult.assertStatusCode(TestRouteResult.scala:136)
	at akka.http.javadsl.testkit.TestRouteResult.assertStatusCode(TestRouteResult.scala:130)
	at io.mantisrx.master.api.akka.route.v1.ResourceClusterNonLeaderRedirectRouteTest.testDisableTaskExecutorsRoute(ResourceClusterNonLeaderRedirectRouteTest.java:202)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:176)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
Caused by: java.lang.AssertionError: Unexpected status code! expected:<200 OK> but was:<500 Internal Server Error>
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotEquals(Assert.java:743)
	at org.junit.Assert.assertEquals(Assert.java:118)
	at akka.http.javadsl.testkit.JUnitRouteTestBase$$anon$1.$anonfun$assertEquals$1(JUnitRouteTest.scala:31)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
	at akka.http.javadsl.testkit.JUnitRouteTestBase$$anon$1.reportDetails(JUnitRouteTest.scala:45)
	... 48 more

There are compile warnings in the code due to:

• deprecated APIs
• missing @Nullable/@NotNull annotations
• incomplete javadocs
• lambda can be replaced with method references
• raw usage of generic types
• unnecessary type parameters (like Map<String> m = new HashMap<String>())
• anonymous class can be replaced with lambdas
• ... and more.

We don't need to solve all of them, but a good coding exercise is to address most those compile warnings when possible. Guidelines are:

1. Not change code logic, which should be addressed and filed PR separately
2. Not change public interfaces (class, methods and fields) that would break other modules or applications. But it's safe to rename private fields or methods when necessary.
3. Prefer readability to more concise code
4. Improve tests in the module is welcome but not required
5. Compile and pass all existing tests.

This tracks the effort for the mantis-runtime module to make the code change scoped. Note this module contains more code than other modules.

CVE-2017-15718 CVE-2018-8009 CVE-2020-9492 CVE-2016-6811 CVE-2018-8029

Recommended upgrade version：2.10.1

Context

Currently, mantis-api remains in its own repository. It would be nice to bring it under the same umbrella.

@calvin681 @sundargates @hmitnflx

Hi dear Netflix developers,
this "issue" has nothing to do with this project however it is of critical importance for you to communicate to your company.
I have a disruptive news for you that is easy to be unaware of since there has been no dedicated public announcement.

Chromium 107 has been released and bring for the first time h265 (GPU) decoding support, on all main OSes.
https://chromestatus.com/feature/5186511939567616
Why does this matter?
Well delivering the optimal codec to the end user is Netflix #1 job.
So you'd better start using it anytime it is supported.
Despite all the ideological hype, AV1 is in many cases not superior to h265 regarding compression ratios but most importantly h265 hardware acceleration is order of magnitudes more widespread. As such it is for end users a significant experience improvement (less heat, less FPS stutter/tearing, MUCH more battery)
Second, it has a gigantic ecological impact, as for you server side it is faster and less resource hungry to compress in h265 but most importantly the significantly reduced electricity [cost]("As you can see a GPU enabled VLC is 70% more energy efficient than using the CPU!" https://devblogs.microsoft.com/sustainable-software/vlc-energy-optimization-with-gpu/#:~:text=As%20you%20can%20see%20a%20GPU%20enabled%20VLC%20is%2070%25%20more%20energy%20efficient%20than%20using%20the%20CPU!) on users hardware will cut the share of internet on electricity consumption worldwide.
Next step is integrating the already released h266!

TL;DR
Crazy high ecological impact and end user impact (battery, smoothness and size)
I hope you can make Netflix realize this and for the company to start using it in the short term.

Context

ServiceRegistry is a singleton instance that holds a reference to MantisPropertiesService. Several components within Mantis use this reference to get certain configuration values at runtime. There are several issues with this approach. Firstly, singletons take a lot of work to manage and test. Secondly, the way the MantisPropertiesService gets loaded is not extendable. We need to consider a more long-term approach to deal with dynamic properties within Mantis. Something like archaius2 might be a good idea.

Goals

1. Getting rid of ServiceRegistry.
2. Clean ways to integrate with various dynamic config implementations.

I've been trying my best to run the PushRequestEventSourceJob in my Mantis development environment. No luck so far. What's the recommended way to accomplish this? Any help would be greatly appreciated. I believe the documentation on this is either out of date or incomplete.

What I've tried so far

1. Uploading via the UI (CORS issue)
2. Adjusting agent build to include the jobs by default (gradle dependency issue)
3. Trying to hack in the artifacts with a docker file (idk, at this point I'm grasping at straws)

Uploading via the UI

I've attempting building the artifacts like so:

./gradlew mantis-source-jobs:mantis-source-job-publish:mantisZipArtifact

But when I drag and drop them into the shared UI, I get CORS errors. I tried installing the https://github.com/Netflix/mantis-ui locally, but it turns out it's a totally different UI vs the one published at https://netflix.github.io/mantis-ui, and it's pre-populated with a bunch of mock data rather than letting me point it at my development Mantis API server.

Adjusting agent build to include the jobs by default

Afaik the SineFunction job is installed via the agent's build.gradle. So I tried adding source-job-publish there as well (and associated dependencies):

def mantisSourcePublishJob = project(":mantis-source-jobs:mantis-source-job-publish")
def mantisSourcePublishJobBuildDir = mantisSourcePublishJob.buildDir
def mantisSourcePublishJobMantisZipArtifact = mantisSourcePublishJob.tasks.named("mantisZipArtifact")

This gives an unexpected error that the task doesn't exist:

A problem occurred evaluating project ':mantis-server:mantis-server-agent'.
> Task with name 'mantisZipArtifact' not found in project ':mantis-source-jobs:mantis-source-job-publish'.

Even though when I call the task directly, it definitely does exist... some kind of dependency ordering issue maybe:

./gradlew :mantis-source-jobs:mantis-source-job-publish:mantisZipArtifact

Trying to hack in the artifacts with a docker file

The agent dockerfile I've come up with looks something like this:

from azul/zulu-openjdk:8-latest as builder
workdir mantis
copy ./mantis .
run ./gradlew mantis-source-jobs:mantis-source-job-publish:mantisZipArtifact

from netflixoss/mantisagent:latest
workdir /apps/mantis/mantis-server-agent
copy --from=builder ./mantis/mantis-source-jobs/mantis-source-job-publish/build/distributions/* ./mantis-artifacts/storage/

The master dockerfile looks something like this (where PushRequestEventSourceJob is essentially a copy of SharedMrePublishEventSource):

from netflixoss/mantiscontrolplaneserver:latest
workdir /apps/mantis/mantis-control-plane-server
copy ./PushRequestEventSourceJob ./job-clusters/.

I end up getting weird service loading errors when I kick-off the job, demonstrating I didn't install it correctly:

2024-01-07 20:41:41 ERROR TaskExecutor:417 - Failed to submit task
java.util.NoSuchElementException
	at java.base/java.util.ServiceLoader$2.next(ServiceLoader.java:1318)
	at java.base/java.util.ServiceLoader$2.next(ServiceLoader.java:1306)
	at java.base/java.util.ServiceLoader$3.next(ServiceLoader.java:1403)
	at io.mantisrx.server.agent.SingleTaskOnlyFactory.getRuntimeTaskInstance(SingleTaskOnlyFactory.java:34)
	at io.mantisrx.server.agent.TaskExecutor.prepareTask(TaskExecutor.java:405)
	at io.mantisrx.server.agent.TaskExecutor.lambda$submitTask$7(TaskExecutor.java:382)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
 	at java.base/java.lang.Thread.run(Thread.java:840)
2024-01-07 20:41:41 INFO  TaskStatusUpdateHandlerImpl:47 - onStatusUpdate for status: {"jobId":"PushRequestEventSourceJob-1","stageNum":1,"workerIndex":0,"workerNumber":1,"type":"INFO","message":"stage 1 worker index=0 number=1 failed during initialization","state":"Failed","hostname":null,"timestamp":1704660101343,"reason":"Normal","payloads":[]}
2024-01-07 20:41:41 ERROR AgentV2Main:65 - Task null failed

Context

Currently, we manually publish docs using the mkdocs gh-deploy command. Ideally, this could be done via CI. The goal of this issue is to achieve that.

CVE-2020-13956

Recommended upgrade version：
4.5.13

Context

There are a lot of unused dependencies in various projects in mantis. The goal of this task is to get rid of them, especially in some of the critical projects such as the runtime.

Steps to reproduce

./gradlew nebulaLint should provide hints on which dependencies are unused in a given module.

Context

Was just reading docs: https://netflix.github.io/mantis/internals/source-jobs/.

Steps to reproduce

Opened the page and seems to have incomplete information. The text refers to four types but only 2 are detailed.

Expected behavior

Either this should read two types of source jobs or explain 4. I'm currently in the process of reading the docs so I do not know which is accurate. I can submit a PR once I know more, but if anyone knows sooner!

Actual Behavior

Explain what actually happens instead.
See screenshot.

There are compile warnings in the code due to:

• deprecated APIs
• missing @Nullable/@NotNull annotations
• incomplete javadocs
• lambda can be replaced with method references
• raw usage of generic types
• unnecessary type parameters (like Map<String> m = new HashMap<String>())
• anonymous class can be replaced with lambdas
• ... and more.

We don't need to solve all of them, but a good coding exercise is to address most those compile warnings when possible. Guidelines are:

1. Not change code logic, which should be addressed and filed PR separately
2. Not change public interfaces (class, methods and fields) that would break other modules or applications. But it's safe to rename private fields or methods when necessary.
3. Prefer readability to more concise code
4. Improve tests in the module is welcome but not required
5. Compile and pass all existing tests.

This tracks the effort for the mantis-connector-iceberg module to make the code change scoped.

Context

I tried to follow your Getting Started and run locally, when I curl from another terminal, I got the "Error registering query" error.

Steps to reproduce

I ran:

curl "localhost:8503?subscriptionId=nj&criterion=select%20country%20from%20stream%20where%20status%3D%3D500&clientId=nj2"

Here is the log:

2019-10-30 11:40:48 INFO TaggedEventFilter:74 - Created SourceEventFilter! for subId [nj2_nj] in Job : UNKNOWN with Id UNKNOWN
2019-10-30 11:40:48 INFO QueryRequestPreProcessor:49 - QueryRequestPreProcessor:queryParams: {subscriptionId=[nj], criterion=[select country from stream where status==500], clientId=[nj2]}
2019-10-30 11:40:48 INFO QueryRequestPreProcessor:60 - Registering query select country from stream where status==500
2019-10-30 11:40:48 INFO QueryRefCountMap:40 - adding query nj2_nj query select country from stream where status==500
2019-10-30 11:40:48 ERROR QueryRequestPreProcessor:68 - Error registering query
java.lang.ExceptionInInitializerError
at io.mantisrx.sourcejob.synthetic.core.MQLQueryManager.registerQuery(MQLQueryManager.java:41)
at io.mantisrx.sourcejob.synthetic.sink.QueryRefCountMap.addQuery(QueryRefCountMap.java:45)
at io.mantisrx.sourcejob.synthetic.sink.QueryRequestPreProcessor.registerQuery(QueryRequestPreProcessor.java:77)
at io.mantisrx.sourcejob.synthetic.sink.QueryRequestPreProcessor.call(QueryRequestPreProcessor.java:62)
at io.mantisrx.sourcejob.synthetic.sink.QueryRequestPreProcessor.call(QueryRequestPreProcessor.java:41)
at io.reactivex.mantis.network.push.PushServerSse$1.handle(PushServerSse.java:254)
at mantis.io.reactivex.netty.protocol.http.server.HttpConnectionHandler$1$1.onNext(HttpConnectionHandler.java:104)
at mantis.io.reactivex.netty.protocol.http.server.HttpConnectionHandler$1$1.onNext(HttpConnectionHandler.java:62)
at rx.subjects.PublishSubject$PublishSubjectProducer.onNext(PublishSubject.java:304)
at rx.subjects.PublishSubject$PublishSubjectState.onNext(PublishSubject.java:219)
at rx.subjects.PublishSubject.onNext(PublishSubject.java:72)
at rx.observers.SerializedObserver.onNext(SerializedObserver.java:91)
at rx.subjects.SerializedSubject.onNext(SerializedSubject.java:67)
at mantis.io.reactivex.netty.pipeline.ObservableAdapter.channelRead(ObservableAdapter.java:50)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at mantis.io.reactivex.netty.protocol.http.server.ServerRequestResponseConverter.channelRead(ServerRequestResponseConverter.java:115)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:241)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at mantis.io.reactivex.netty.metrics.BytesInspector.channelRead(BytesInspector.java:59)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:138)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Attempting to call unbound fn: #'io.mantisrx.mql.shaded.clojure.com.netflix.mantis.examples.core/require
at io.mantisrx.mql.shaded.clojure.lang.Var$Unbound.throwArity(Var.java:43)
at io.mantisrx.mql.shaded.clojure.lang.AFn.invoke(AFn.java:32)
at io.mantisrx.mql.shaded.clojure.lang.Var.invoke(Var.java:379)
at io.mantisrx.sourcejob.synthetic.core.MQL.(MQL.java:54)
... 52 more

Expected behavior

Should return some matching events.

Actual Behavior

Throw exception. May be I need to use some meaningful subscriptionId and clientId?

Context

This is similar to #384 but from the context of the WorkerManager.

Expected behavior

All loglines emitted by the WorkerManager contain the WorkerID they are responsible for.

Summary

Hey folks. I'm encountering crashing containers while following the official docker-compose guide getting started guide:

$ mkdir mantis
$ cd mantis
$ wget https://raw.githubusercontent.com/Netflix/mantis/master/docker-compose.yml
$ docker-compose -f docker-compose.yml up

I can see logs showing that mantisagent, mantismaster, and mantisapi all exit with non-zero status codes (in that order):

mantis-compose-mantisagent-1 exited with code 2
mantis-compose-mantismaster-1 exited with code 1
mantis-compose-mantisapi-1 exited with code 1

I'm running this on OSX Sonoma (aarch64), but have been able to reproduce on Linux (amd64) as well.

Details

Right before the crash of the agent and the master, I see the following log line from ZooKeeper:

mantis-compose-zookeeper-1         | 2023-12-13 01:40:04,582 [myid:] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@675] - Received packet at server of unknown type 19

Other potential culprit logs (there are too many to dump all of them here) include:

mantis-compose-mantisagent-1       | 2023-12-13 01:40:03 ERROR AgentV2Main:114 - Unexpected error: java.lang.IllegalStateException: Expected the service TaskExecutorStarter [FAILED] to be RUNNING, but the service has FAILED

mantis-compose-mantisagent-1       | Caused by: io.mantisrx.shaded.org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for /mantis/master

mantis-compose-mantismaster-1      | 2023-12-13 01:40:04 ERROR MasterMain:269 - caught exception on Mantis Master initialization
mantis-compose-mantismaster-1      | java.lang.RuntimeException: java.lang.IllegalStateException: Expected the service ZookeeperMasterMonitor [FAILED] to be RUNNING, but the service has FAILED

mantis-compose-mantismaster-1      | Caused by: io.mantisrx.shaded.org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for /mantis/master

Context

Explain symptoms and other details for the issue.

Steps to reproduce

Explain the steps to reliably reproduce the issue.

Expected behavior

Explain what you think should happen.

Actual Behavior

Explain what actually happens instead.

There are compile warnings in the code due to:

• deprecated APIs
• missing @Nullable/@NotNull annotations
• incomplete javadocs
• lambda can be replaced with method references
• raw usage of generic types
• unnecessary type parameters (like Map<String> m = new HashMap<String>())
• anonymous class can be replaced with lambdas
  and more.

We don't need to solve all of them, but a good coding exercise is to address most those compile warnings when possible. Guidelines are:

1. Not change code logic, which should be addressed and filed PR separately
2. Not change public interfaces (class, methods and fields) that would break other modules or applications. But it's safe to rename private fields or methods when necessary.
3. Prefer readability to more concise code
4. Improve tests in the module is welcome but not required
5. Compile and pass all existing tests.

This tracks the effort for the mantis-control-plane-client module to make the code change scoped.

Context

We want to be on the latest and greatest version of junit - JUnit 5.

Context

mantis-examples-synthetic-sourcejob does not work.

Steps to reproduce

Explain the steps to reliably reproduce the issue.

Expected behavior

runs successfully without issues.

Actual Behavior

Exception in thread "main" java.lang.NoClassDefFoundError: com/netflix/spectator/api/Registry

Caused by: java.lang.ClassNotFoundException: com.netflix.spectator.api.Registry

Build Failing with Gradle 8.0

Steps to reproduce

Run the command ./gradlew clean build

Expected behavior

BUILD SUCCESS

Actual Behavior

Task :mantis-common:compileJava FAILED
warning: [options] source value 8 is obsolete and will be removed in a future release
warning: [options] target value 8 is obsolete and will be removed in a future release
warning: [options] To suppress warnings about obsolete options, use -Xlint:-options.
//mantis/mantis-common/src/main/java/io/reactivx/mantis/operators/OperatorOnErrorResumeNextViaFunction.java:76: warning: [deprecation] getInstance() in RxJavaPlugins has been deprecated
RxJavaPlugins.getInstance().getErrorHandler().handleError(e);
^
//mantis/mantis-common/src/main/java/io/reactivx/mantis/operators/OperatorOnErrorResumeNextViaFunction.java:76: warning: [deprecation] handleError(Throwable) in RxJavaErrorHandler has been deprecated
RxJavaPlugins.getInstance().getErrorHandler().handleError(e);
^
//mantis/mantis-common/src/main/java/io/reactivx/mantis/operators/OnSubscribeRedo.java:79: warning: [deprecation] create(OnSubscribe) in Observable has been deprecated
return create(new OnSubscribeRedo(source, notificationHandler, true, false, Schedulers.trampoline()));
^
where T is a type-variable:
T extends Object declared in method create(OnSubscribe)
//mantis/mantis-common/src/main/java/io/reactivx/mantis/operators/OnSubscribeRedo.java:83: warning: [deprecation] create(OnSubscribe) in Observable has been deprecated
return create(new OnSubscribeRedo(source, notificationHandler, true, false, scheduler));
^
where T is a type-variable:
T extends Object declared in method create(OnSubscribe)
//mantis/mantis-common/src/main/java/io/reactivx/mantis/operators/OnSubscribeRedo.java:108: warning: [deprecation] create(OnSubscribe) in Observable has been deprecated
return create(new OnSubscribeRedo(source, notificationHandler, false, true, Schedulers.trampoline()));
^
where T is a type-variable:
T extends Object declared in method create(OnSubscribe)
//mantis/mantis-common/src/main/java/io/reactivx/mantis/operators/OnSubscribeRedo.java:112: warning: [deprecation] create(OnSubscribe) in Observable has been deprecated
return create(new OnSubscribeRedo(source, notificationHandler, false, true, scheduler));
^
where T is a type-variable:
T extends Object declared in method create(OnSubscribe)
/~/mantis/mantis-common/src/main/java/io/reactivx/mantis/operators/OnSubscribeRedo.java:116: warning: [deprecation] create(OnSubscribe) in Observable has been deprecated
return create(new OnSubscribeRedo(source, notificationHandler, false, false, scheduler));
^
where T is a type-variable:
T extends Object declared in method create(OnSubscribe)
10 warnings
An exception has occurred in the compiler (20.0.2). Please file a bug against the Java compiler via the Java bug reporting page (https://bugreport.java.com) after checking the Bug Database (https://bugs.java.com) for duplicates. Include your program, the following diagnostic, and the parameters passed to the Java compiler in your report. Thank you.
java.lang.ClassCastException: class com.sun.tools.javac.tree.JCTree$JCLiteral cannot be cast to class com.sun.tools.javac.tree.JCTree$JCCaseLabel (com.sun.tools.javac.tree.JCTree$JCLiteral and com.sun.tools.javac.tree.JCTree$JCCaseLabel are in module jdk.compiler of loader 'app')
at java.base/java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
at java.base/java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1932)
at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:280)
at java.base/java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1932)
at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:129)
at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:527)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:513)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
at java.base/java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.anyMatch(ReferencePipeline.java:632)
at jdk.compiler/com.sun.tools.javac.comp.Attr.handleSwitch(Attr.java:1696)
at jdk.compiler/com.sun.tools.javac.comp.Attr.visitSwitch(Attr.java:1608)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCSwitch.accept(JCTree.java:1305)
at jdk.compiler/com.sun.tools.javac.comp.Attr.attribTree(Attr.java:663)
at jdk.compiler/com.sun.tools.javac.comp.Attr.attribStat(Attr.java:737)
at jdk.compiler/com.sun.tools.javac.comp.Attr.attribStats(Attr.java:756)
at jdk.compiler/com.sun.tools.javac.comp.Attr.visitBlock(Attr.java:1440)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCBlock.accept(JCTree.java:1088)
at jdk.compiler/com.sun.tools.javac.comp.Attr.attribTree(Attr.java:663)
at jdk.compiler/com.sun.tools.javac.comp.Attr.attribStat(Attr.java:737)
at jdk.compiler/com.sun.tools.javac.comp.Attr.visitMethodDef(Attr.java:1230)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(JCTree.java:912)
at jdk.compiler/com.sun.tools.javac.comp.Attr.attribTree(Attr.java:663)
at jdk.compiler/com.sun.tools.javac.comp.Attr.attribStat(Attr.java:737)
at jdk.compiler/com.sun.tools.javac.comp.Attr.attribClassBody(Attr.java:5592)
at jdk.compiler/com.sun.tools.javac.comp.Attr.attribClass(Attr.java:5485)
at jdk.compiler/com.sun.tools.javac.comp.Attr.attribClass(Attr.java:5309)
at jdk.compiler/com.sun.tools.javac.comp.Attr.visitClassDef(Attr.java:971)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:810)
at jdk.compiler/com.sun.tools.javac.comp.Attr.attribTree(Attr.java:663)
at jdk.compiler/com.sun.tools.javac.comp.Attr.attribStat(Attr.java:737)
at jdk.compiler/com.sun.tools.javac.comp.Attr.attribClassBody(Attr.java:5592)
at jdk.compiler/com.sun.tools.javac.comp.Attr.attribClass(Attr.java:5485)
at jdk.compiler/com.sun.tools.javac.comp.Attr.attribClass(Attr.java:5309)
at jdk.compiler/com.sun.tools.javac.comp.Attr.attrib(Attr.java:5248)
at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.attribute(JavaCompiler.java:1331)
at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:960)
at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.lambda$doCall$0(JavacTaskImpl.java:104)
at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.invocationHelper(JavacTaskImpl.java:152)
at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:100)
at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:94)
at org.gradle.internal.compiler.java.IncrementalCompileTask.call(IncrementalCompileTask.java:89)
at org.gradle.api.internal.tasks.compile.AnnotationProcessingCompileTask.call(AnnotationProcessingCompileTask.java:94)
at org.gradle.api.internal.tasks.compile.ResourceCleaningCompilationTask.call(ResourceCleaningCompilationTask.java:57)
at org.gradle.api.internal.tasks.compile.JdkJavaCompiler.execute(JdkJavaCompiler.java:54)
at org.gradle.api.internal.tasks.compile.JdkJavaCompiler.execute(JdkJavaCompiler.java:39)
at org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler.delegateAndHandleErrors(NormalizingJavaCompiler.java:97)
at org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler.execute(NormalizingJavaCompiler.java:51)
at org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler.execute(NormalizingJavaCompiler.java:37)
at org.gradle.api.internal.tasks.compile.AnnotationProcessorDiscoveringCompiler.execute(AnnotationProcessorDiscoveringCompiler.java:51)
at org.gradle.api.internal.tasks.compile.AnnotationProcessorDiscoveringCompiler.execute(AnnotationProcessorDiscoveringCompiler.java:37)
at org.gradle.api.internal.tasks.compile.ModuleApplicationNameWritingCompiler.execute(ModuleApplicationNameWritingCompiler.java:46)
at org.gradle.api.internal.tasks.compile.ModuleApplicationNameWritingCompiler.execute(ModuleApplicationNameWritingCompiler.java:36)
at org.gradle.jvm.toolchain.internal.DefaultToolchainJavaCompiler.execute(DefaultToolchainJavaCompiler.java:57)
at org.gradle.api.tasks.compile.JavaCompile.lambda$createToolchainCompiler$1(JavaCompile.java:232)
at org.gradle.api.internal.tasks.compile.CleaningJavaCompiler.execute(CleaningJavaCompiler.java:53)
at org.gradle.api.internal.tasks.compile.incremental.IncrementalCompilerFactory.lambda$createRebuildAllCompiler$0(IncrementalCompilerFactory.java:52)
at org.gradle.api.internal.tasks.compile.incremental.SelectiveCompiler.execute(SelectiveCompiler.java:67)
at org.gradle.api.internal.tasks.compile.incremental.SelectiveCompiler.execute(SelectiveCompiler.java:41)
at org.gradle.api.internal.tasks.compile.incremental.IncrementalResultStoringCompiler.execute(IncrementalResultStoringCompiler.java:66)
at org.gradle.api.internal.tasks.compile.incremental.IncrementalResultStoringCompiler.execute(IncrementalResultStoringCompiler.java:52)
at org.gradle.api.internal.tasks.compile.CompileJavaBuildOperationReportingCompiler$2.call(CompileJavaBuildOperationReportingCompiler.java:59)
at org.gradle.api.internal.tasks.compile.CompileJavaBuildOperationReportingCompiler$2.call(CompileJavaBuildOperationReportingCompiler.java:51)
at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73)
at org.gradle.api.internal.tasks.compile.CompileJavaBuildOperationReportingCompiler.execute(CompileJavaBuildOperationReportingCompiler.java:51)
at org.gradle.api.tasks.compile.JavaCompile.performCompilation(JavaCompile.java:279)
at org.gradle.api.tasks.compile.JavaCompile.performIncrementalCompilation(JavaCompile.java:165)
at org.gradle.api.tasks.compile.JavaCompile.compile(JavaCompile.java:146)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
at java.base/java.lang.reflect.Method.invoke(Method.java:578)
at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:125)
at org.gradle.api.internal.project.taskfactory.IncrementalInputsTaskAction.doExecute(IncrementalInputsTaskAction.java:32)
at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:51)
at org.gradle.api.internal.project.taskfactory.AbstractIncrementalTaskAction.execute(AbstractIncrementalTaskAction.java:25)
at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:29)
at org.gradle.api.internal.tasks.execution.TaskExecution$3.run(TaskExecution.java:236)
at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29)
at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
at org.gradle.internal.operations.DefaultBuildOperationRunner.run(DefaultBuildOperationRunner.java:47)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:68)
at org.gradle.api.internal.tasks.execution.TaskExecution.executeAction(TaskExecution.java:221)
at org.gradle.api.internal.tasks.execution.TaskExecution.executeActions(TaskExecution.java:204)
at org.gradle.api.internal.tasks.execution.TaskExecution.executeWithPreviousOutputFiles(TaskExecution.java:187)
at org.gradle.api.internal.tasks.execution.TaskExecution.execute(TaskExecution.java:165)
at org.gradle.internal.execution.steps.ExecuteStep.executeInternal(ExecuteStep.java:89)
at org.gradle.internal.execution.steps.ExecuteStep.access$000(ExecuteStep.java:40)
at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:53)
at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:50)
at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73)
at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:50)
at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:40)
at org.gradle.internal.execution.steps.RemovePreviousOutputsStep.execute(RemovePreviousOutputsStep.java:68)
at org.gradle.internal.execution.steps.RemovePreviousOutputsStep.execute(RemovePreviousOutputsStep.java:38)
at org.gradle.internal.execution.steps.CancelExecutionStep.execute(CancelExecutionStep.java:41)
at org.gradle.internal.execution.steps.TimeoutStep.executeWithoutTimeout(TimeoutStep.java:74)
at org.gradle.internal.execution.steps.TimeoutStep.execute(TimeoutStep.java:55)
at org.gradle.internal.execution.steps.CreateOutputsStep.execute(CreateOutputsStep.java:51)
at org.gradle.internal.execution.steps.CreateOutputsStep.execute(CreateOutputsStep.java:29)
at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.executeDelegateBroadcastingChanges(CaptureStateAfterExecutionStep.java:124)
at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.execute(CaptureStateAfterExecutionStep.java:80)
at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.execute(CaptureStateAfterExecutionStep.java:58)
at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:48)
at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:36)
at org.gradle.internal.execution.steps.BuildCacheStep.executeWithoutCache(BuildCacheStep.java:181)
at org.gradle.internal.execution.steps.BuildCacheStep.lambda$execute$1(BuildCacheStep.java:71)
at org.gradle.internal.Either$Right.fold(Either.java:175)
at org.gradle.internal.execution.caching.CachingState.fold(CachingState.java:59)
at org.gradle.internal.execution.steps.BuildCacheStep.execute(BuildCacheStep.java:69)
at org.gradle.internal.execution.steps.BuildCacheStep.execute(BuildCacheStep.java:47)
at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:36)
at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:25)
at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:36)
at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:22)
at org.gradle.internal.execution.steps.SkipUpToDateStep.executeBecause(SkipUpToDateStep.java:110)
at org.gradle.internal.execution.steps.SkipUpToDateStep.lambda$execute$2(SkipUpToDateStep.java:56)
at java.base/java.util.Optional.orElseGet(Optional.java:364)
at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:56)
at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:38)
at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:73)
at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:44)
at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:37)
at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:27)
at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:89)
at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:50)
at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:114)
at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:57)
at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:76)
at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:50)
at org.gradle.internal.execution.steps.SkipEmptyWorkStep.executeWithNoEmptySources(SkipEmptyWorkStep.java:254)
at org.gradle.internal.execution.steps.SkipEmptyWorkStep.executeWithNoEmptySources(SkipEmptyWorkStep.java:209)
at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:88)
at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:56)
at org.gradle.internal.execution.steps.RemoveUntrackedExecutionStateStep.execute(RemoveUntrackedExecutionStateStep.java:32)
at org.gradle.internal.execution.steps.RemoveUntrackedExecutionStateStep.execute(RemoveUntrackedExecutionStateStep.java:21)
at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsStartedStep.execute(MarkSnapshottingInputsStartedStep.java:38)
at org.gradle.internal.execution.steps.LoadPreviousExecutionStateStep.execute(LoadPreviousExecutionStateStep.java:43)
at org.gradle.internal.execution.steps.LoadPreviousExecutionStateStep.execute(LoadPreviousExecutionStateStep.java:31)
at org.gradle.internal.execution.steps.AssignWorkspaceStep.lambda$execute$0(AssignWorkspaceStep.java:40)
at org.gradle.api.internal.tasks.execution.TaskExecution$4.withWorkspace(TaskExecution.java:281)
at org.gradle.internal.execution.steps.AssignWorkspaceStep.execute(AssignWorkspaceStep.java:40)
at org.gradle.internal.execution.steps.AssignWorkspaceStep.execute(AssignWorkspaceStep.java:30)
at org.gradle.internal.execution.steps.IdentityCacheStep.execute(IdentityCacheStep.java:37)
at org.gradle.internal.execution.steps.IdentityCacheStep.execute(IdentityCacheStep.java:27)
at org.gradle.internal.execution.steps.IdentifyStep.execute(IdentifyStep.java:44)
at org.gradle.internal.execution.steps.IdentifyStep.execute(IdentifyStep.java:33)
at org.gradle.internal.execution.impl.DefaultExecutionEngine$1.execute(DefaultExecutionEngine.java:76)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:139)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:128)
at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:77)
at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:51)
at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:69)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:327)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:314)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:307)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:293)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:417)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:339)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1623)

FAILURE: Build failed with an exception.

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

There are compile warnings in the code due to:

• deprecated APIs
• missing @Nullable/@NotNull annotations
• incomplete javadocs
• lambda can be replaced with method references
• raw usage of generic types
• unnecessary type parameters (like Map<String> m = new HashMap<String>())
• anonymous class can be replaced with lambdas
• ... and more.

We don't need to solve all of them, but a good coding exercise is to address most those compile warnings when possible. Guidelines are:

1. Not change code logic, which should be addressed and filed PR separately
2. Not change public interfaces (class, methods and fields) that would break other modules or applications. But it's safe to rename private fields or methods when necessary.
3. Prefer readability to more concise code
4. Improve tests in the module is welcome but not required
5. Compile and pass all existing tests.

This tracks the effort for the mantis-control-plane-server module to make the code change scoped.

Context

We want mantis to be on Java17. Unfortunately, this would require several gradle changes. So, the goal of this task is to make these changes and ensure the builds succeed.

Summary

Hey folks. I'm having some difficulty building the complete Mantis project, as well as its examples from source.

I assume this is my environment being incorrectly configured rather than an issue with the master branch.

It would be awesome if we could figure out what's missing and update the docs to include those dependencies.

Thanks for taking a look!

Reproduction

I get errors when following the build instructions in the README:

$ git clone [email protected]:Netflix/mantis.git
$ cd mantis
$ ./gradlew clean build
BUILD FAILED in 4s

I get similar errors when following the instructions from the "Local" tutorial:

$ git clone [email protected]:Netflix/mantis.git
$ cd mantis/mantis-examples/mantis-examples-synthetic-sourcejob
$ ../../gradlew execute
BUILD FAILED in 3s

I've demonstrated my environment is at least somewhat correctly setup by successfully building mantis-mql from source:

$ git clone https://github.com/Netflix/mantis-mql
$ ./gradlew clean build
BUILD SUCCESSFUL in 52s

Here's some metadata around what I believe is being used to build Mantis:

$ ./gradlew --version

------------------------------------------------------------
Gradle 7.5
------------------------------------------------------------

Build time:   2022-07-14 12:48:15 UTC
Revision:     c7db7b958189ad2b0c1472b6fe663e6d654a5103

Kotlin:       1.6.21
Groovy:       3.0.10
Ant:          Apache Ant(TM) version 1.10.11 compiled on July 10 2021
JVM:          17.0.6 (Eclipse Adoptium 17.0.6+10)
OS:           Mac OS X 14.1.2 aarch64

Details

It looks like at least two tasks are failing with quite loud stacktraces which I'm partially summarizing below:

> Task :mantis-common-serde:spotlessJava FAILED
Step 'removeUnusedImports' found problem in 'mantis-common-serde/src/main/java/io/mantisrx/common/JsonSerializer.java':
null
java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
...
        at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.IllegalAccessError: class com.google.googlejavaformat.java.RemoveUnusedImports (in unnamed module @0x2c6b2f25) cannot access class com.sun.tools.javac.util.Context (in module jdk.compiler) because module jdk.compiler does not export com.sun.tools.javac.util to unnamed module @0x2c6b2f25
        at com.google.googlejavaformat.java.RemoveUnusedImports.removeUnusedImports(RemoveUnusedImports.java:188)
        ... 136 more

> Task :mantis-common:compileJava FAILED
/Users/jpittis/mantis/mantis-common/src/main/java/io/reactivx/mantis/operators/OperatorOnErrorResumeNextViaFunction.java:76: warning: [deprecation] getInstance() in RxJavaPlugins has been deprecated
                    RxJavaPlugins.getInstance().getErrorHandler().handleError(e);
                                 ^
/Users/jpittis/mantis/mantis-common/src/main/java/io/reactivx/mantis/operators/OperatorOnErrorResumeNextViaFunction.java:76: warning: [deprecation] handleError(Throwable) in RxJavaErrorHandler has been deprecated
                    RxJavaPlugins.getInstance().getErrorHandler().handleError(e);
                                                                 ^
/Users/jpittis/mantis/mantis-common/src/main/java/io/reactivx/mantis/operators/OnSubscribeRedo.java:79: warning: [deprecation] <T>create(OnSubscribe<T>) in Observable has been deprecated
        return create(new OnSubscribeRedo<T>(source, notificationHandler, true, false, Schedulers.trampoline()));
               ^
  where T is a type-variable:
    T extends Object declared in method <T>create(OnSubscribe<T>)
/Users/jpittis/mantis/mantis-common/src/main/java/io/reactivx/mantis/operators/OnSubscribeRedo.java:83: warning: [deprecation] <T>create(OnSubscribe<T>) in Observable has been deprecated
        return create(new OnSubscribeRedo<T>(source, notificationHandler, true, false, scheduler));
               ^
  where T is a type-variable:
    T extends Object declared in method <T>create(OnSubscribe<T>)
/Users/jpittis/mantis/mantis-common/src/main/java/io/reactivx/mantis/operators/OnSubscribeRedo.java:108: warning: [deprecation] <T>create(OnSubscribe<T>) in Observable has been deprecated
        return create(new OnSubscribeRedo<T>(source, notificationHandler, false, true, Schedulers.trampoline()));
               ^
  where T is a type-variable:
    T extends Object declared in method <T>create(OnSubscribe<T>)
/Users/jpittis/mantis/mantis-common/src/main/java/io/reactivx/mantis/operators/OnSubscribeRedo.java:112: warning: [deprecation] <T>create(OnSubscribe<T>) in Observable has been deprecated
        return create(new OnSubscribeRedo<T>(source, notificationHandler, false, true, scheduler));
               ^
  where T is a type-variable:
    T extends Object declared in method <T>create(OnSubscribe<T>)
/Users/jpittis/mantis/mantis-common/src/main/java/io/reactivx/mantis/operators/OnSubscribeRedo.java:116: warning: [deprecation] <T>create(OnSubscribe<T>) in Observable has been deprecated
        return create(new OnSubscribeRedo<T>(source, notificationHandler, false, false, scheduler));
               ^
  where T is a type-variable:
    T extends Object declared in method <T>create(OnSubscribe<T>)
7 warnings
An exception has occurred in the compiler (17.0.6). Please file a bug against the Java compiler via the Java bug reporting page (https://bugreport.java.com) after checking the Bug Database (https://bugs.java.com) for duplicates. Include your program, the following diagnostic, and the parameters passed to the Java compiler in your report. Thank you.
java.lang.NullPointerException: Cannot read field "bindingsWhenTrue" because "currentBindings" is null
        at jdk.compiler/com.sun.tools.javac.comp.Attr.handleSwitch(Attr.java:1794)
...

Context

Upgrade the whole repo to use Mockito version 3+.
Currently, there are breaks in the existing code base when applying Mockito 3+.