flipkart-incubator / phantom

Phantom is a high performance proxy for accessing distributed services. It is an RPC system with support for different transports and protocols. Phantom is inspired by Twitter Finagle clients and builds on the capabilities of technologies like Netty, Unix Domain Sockets, Netflix Hystrix and Spring. Phantom proxies have been used to serve several hundred million API calls in production deployments at Flipkart.

Phantom

Phantom is a high performance proxy for accessing distributed services. It is an RPC system with support for different transports and protocols. Phantom is inspired by Twitter Finagle clients and builds on the capabilities of technologies like Netty, Unix Domain Sockets, Netflix Hystrix and Spring. Phantom proxies have been used to serve several hundred million API calls in production deployments at Flipkart.

Releases

Release	Date	Description
Version 3.8.0	Dec 2021	Upgrade log4j to 2.16.0
Version 3.7.0	Dec 2021	Upgrade log4j to 2.15.0
Version 3.6.0	Nov 2021	Filtering commands and threadpools in hystrix dashboard
Version 3.5.0	Sep 2020	Spring 5 Upgrade & Corresponding XML updates
Version 3.4.0	May 2017	Passing execution errors to fallback
Version 3.3.0	Feb 2017	Upgrading to Jetty 9.4.1.v20170120
Version 3.2.0	Jan 2017	Hystrix 1.5.8, Customizable core and max thread pool sizes for Task

Changelog

Changelog can be viewed in CHANGELOG.md file (https://github.com/Flipkart/phantom/blob/master/CHANGELOG.md)

Why Phantom

Phantom is the "ghost who walks" - an entity whose presence can be felt (in a good way) but its existence need not be acknowledged. The Service Proxies fit this analogy well and therefore earned the moniker "Phantom". Motivation for creating Phantom and design overview is described in this Proxies for resilience and fault tolerance in SOA blog post.

Phantom Consoles

Getting Started

The Getting Started page has "5 minute" examples to help you start using Phantom proxies.

Documentation and Examples

Phantom project modules that start with "sample" - for e.g. sample-http-proxy, sample-task-proxy are example proxy implementations. Documentation is continuously being added to the Wiki page of Phantom (https://github.com/Flipkart/phantom/wiki)

Getting help

For discussion, help regarding usage, or receiving important announcements, subscribe to the Phantom users mailing list: http://groups.google.com/group/phantom-users

License

Phantom is licensed under : The Apache Software License, Version 2.0. Here is a copy of the license (http://www.apache.org/licenses/LICENSE-2.0.txt)

Core contributors

• Regunath B (@regunathb)
• Devashish Shankar (@devashishshankar)
• Kartik B Ukhalkar (@kartikssj)
• Arya Ketan (@aryaKetan)
• Amanpreet Singh (@aman_high)

phantom's Issues

Benchmark Suite

I was not able to find a benchmarking suite for this. If there is something already available, please point me to that direction. What I am looking for is a simple suite that spins up minimal services of all/some protocols and benchmarks them against a phantom instance. Added chaos can help see how it responds to error.

If there are already published benchmarks somewhere, that can help too.

Thanks.

how does phantom support cluster?

does phantom support in the distrubute environment?anything you talk, I would be very grateful!.thank you .

Upgrading to netty 4

Can we upgrade it to netty 4 as it has major performance improvements.

Async command execution does not log errors if the command fails.

Test Issue. Do Not Delete.

Maven build is failing

It appears to be unable to locate junixsocket which is hosted under Goolge Code.

I do not know maven well; attempts to currect this shortcoming is getting me deeper and deeper.

That library is assiciated with the package: org.newsclub.net.unix

[INFO] Building Netty/UDS 1.2.9-SNAPSHOT
...
[INFO] Reactor Summary:
[INFO]
[INFO] Phantom Full Build ................................ SUCCESS [ 0.284 s]
[INFO] Task .............................................. SUCCESS [ 1.784 s]
[INFO] Thrift Task ....................................... SUCCESS [ 0.876 s]
[INFO] HTTP Task ......................................... SUCCESS [ 0.777 s]
[INFO] Phantom Runtime ................................... SUCCESS [ 2.985 s]
[INFO] Netty/UDS ......................................... FAILURE [ 0.477 s]
[INFO] OIO UDS Server .................................... SKIPPED
...
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.0.2:compile (default-compile) on project netty-uds: Compilation failure: Compilation failure:
[ERROR] /home/mdykman/projects/phantom/netty-uds/src/main/java/com/flipkart/phantom/netty/uds/OioServerSocketPipelineSink.java:[24,28] error: package org.newsclub.net.unix does not exist

how to use phantom with thrift tframetransport

Hi,
phantom is really a nice middle-ware to proxy and monitor rpc calls.
But when the phantom comes with tframetransport, it doesn't work as expected.
Could give me some ideas how to make it work.

Thanks.

Maven dependency

Hi folks

I find that there is a missing artifact issue problem on junixsocket. To solve this issue, I manually download the artifact from https://code.google.com/p/junixsocket/ to my local maven repository. Hope this inconvenience can be fixed.

Tracing use cases

Hi,

I am writing a book on Distributed tracing for O'Reilly. I am currently researching on how companies use tracing at Flipkart so I can include it my book. Are there any pointers or talks on how Phantom tracing is used at Flipkart?

Thanks.

the JettyWebAppContextFactory replace the first and not the last occurrence

I'm bundling phantom in war (just to make start and stop easier for ops)

One thing that blocks me is that the JetteyWebAppContextFactory does'nt work well when the path contains WEB-INF

eg
....workspace/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/wtpwebapps/phantom-war/WEB-INF/lib/runtime-1.4.4.jar!/WEB-INF
is transformed to
...workspace/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/wtpwebapps/phantom-war//lib/runtime-1.4.4.jar!/WEB-INF
and not
...workspace/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/wtpwebapps/phantom-war/WEB-INF/lib/runtime-1.4.4.jar!/

The code looks like

// trim off the "WEB-INF" part as the WebAppContext path should refer to the parent directory
        if (path.endsWith("WEB-INF")) {
            path = path.replace("WEB-INF", "");
        }

is it possible to modify it to something like

    // trim off the "WEB-INF" part as the WebAppContext path should refer to the parent directory
        if (path.endsWith("WEB-INF")) {
            path = path.substring(0, path.length() - "WEB-INF".length());
        }

or is there a way to override the common-proxy-server-nature-config.xml with my own definition ?

Phantom 1.3.6 release depends on Trooper 1.3.2-SNAPSHOT

Proxy CONNECT Aborted

On Client Side

MT-C02HL1FYDV16:sample-http-proxy srikanth.adimulam$ curl --proxy "http://localhost:8080" https://seller.flipkart.com
curl: (56) Proxy CONNECT aborted

On proxy terminal

10:17:11.861 [New I/O server worker #1-1] ERROR c.f.p.r.i.s.n.h.h.RoutingHttpChannelHandler - Exception thrown on Channel. Disconnect initiated : java.lang.NullPointerException: Span name should not be null.
java.lang.NullPointerException: Span name should not be null.
at org.apache.commons.lang3.Validate.notBlank(Validate.java:448) ~[commons-lang3-3.1.jar:3.1]
at com.github.kristofa.brave.ServerTracerImpl.setStateUnknown(ServerTracerImpl.java:92) ~[brave-impl-2.2.1.jar:na]
at com.flipkart.phantom.task.impl.interceptor.ServerRequestInterceptor.process(ServerRequestInterceptor.java:71) ~[task-1.4.2-SNAPSHOT.jar:na]
at com.flipkart.phantom.runtime.impl.server.netty.handler.http.RoutingHttpChannelHandler.initializeServerTracing(RoutingHttpChannelHandler.java:295) ~[channel-handler-http-1.4.2-SNAPSHOT.jar:na]
at com.flipkart.phantom.runtime.impl.server.netty.handler.http.RoutingHttpChannelHandler.messageReceived(RoutingHttpChannelHandler.java:213) ~[channel-handler-http-1.4.2-SNAPSHOT.jar:na]
at org.jboss.netty.handler.codec.http.HttpChunkAggregator.messageReceived(HttpChunkAggregator.java:111) ~[netty-3.3.1.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) ~[netty-3.3.1.Final.jar:na]
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.unfoldAndFireMessageReceived(ReplayingDecoder.java:522) ~[netty-3.3.1.Final.jar:na]
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:501) ~[netty-3.3.1.Final.jar:na]
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:438) ~[netty-3.3.1.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) ~[netty-3.3.1.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) ~[netty-3.3.1.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:343) ~[netty-3.3.1.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:274) ~[netty-3.3.1.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:194) ~[netty-3.3.1.Final.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_60]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_60]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]

Phantom sample-http-proxy not running with error in bootstrap sequence

While running phantom sample-http-proxy there are few issues after successful build of Phantom with different version of Trooper dependencies:

Running with the Command given in getting started:

java -cp "./target/lib/*" org.trpr.platform.runtime.impl.bootstrap.BootstrapLauncher ./src/main/resources/external/bootstrap.xml

With org.trpr.version - 1.3.3

1. First Error -- Error initializing logging framework!. Error message : org.slf4j.impl.Log4jLoggerFactory cannot be cast to ch.qos.logback.classic.LoggerContext
   java.lang.ClassCastException: org.slf4j.impl.Log4jLoggerFactory cannot be cast to ch.qos.logback.classic.LoggerContext

Solved: I solved it by manually removing slf4j-log4j12-1.7.5.jar from target/lib of sample-http-proxy.
But it should be excluded from POM.

2. Second Error : Bean property 'componentContainerClassName' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?

Solved: I solved it by commenting out property componentContainerClassName in bootstrap.xml as i checked in the source code of Trooper and didnot find the attribute in Bootstrap class

https://github.com/regunathb/Trooper/blob/master/runtime-core/src/main/java/org/trpr/platform/runtime/spi/bootstrap/BootstrapInfo.java

After fixing this two issues, im able to run properly,here's the output

** Trooper runtime Bootstrap start **
Using logging config file : /home/kuntal/knowledge/bigdata/phantom/sample-http-proxy/./src/main/resources/external/logback.xml
16:58:15.834 [main] INFO o.t.p.r.i.bootstrap.spring.Bootstrap - Logging framework is now initialized!
16:58:15.943 [main] INFO o.s.c.s.FileSystemXmlApplicationContext - Refreshing org.springframework.context.support.FileSystemXmlApplicationContext@73871392: startup date [Sun Mar 20 16:58:15 IST 2016]; root of context hierarchy
16:58:15.983 [main] INFO o.s.b.f.xml.XmlBeanDefinitionReader - Loading XML bean definitions from URL [file:/home/kuntal/knowledge/bigdata/phantom/sample-http-proxy/./src/main/resources/external/bootstrap.xml]
16:58:16.113 [main] INFO o.t.p.r.i.c.s.SpringContainerImpl - Initializing Bootstrap extension .... ~~~TrprPlatformCommonBeansContext~~~
16:58:16.113 [main] INFO o.t.p.r.i.c.s.SpringContainerImpl - No component container configured for this runtime instance.
16:58:16.115 [main] INFO o.t.p.r.i.bootstrap.spring.Bootstrap -

---

Trooper __
**/ \ Runtime Nature : SERVER
__/ **/ Component Container : None
/ __/ \ Startup Time : 409 ms
__/ __/ Host Name: kuntal-Lenovo-G500
__/

---

16:58:16.115 [main] INFO o.t.p.r.i.bootstrap.spring.Bootstrap - ** Trooper Bootstrap complete **

But now when i'm trying to run the below commands, nothing is happening:

http://localhost:8080
http://localhost:8081/admin/dashboard
http://localhost:8081/admin/configuration

I checked with *netstat * command and nothing is listening on this ports.

• Please tell me what to do in-order to make my proxy work and redirect to Google.com as configured in spring proxy handler config???

Also i tried with With org.trpr.version - 1.3.4

I faced the same issues of the above ( logback cast exception and spring invalid attributes)
Logback issue was fixed similar to above, but even after commenting the spring attributes, im getting this error:

* Trooper runtime Bootstrap start **
Using logging config file : /home/kuntal/knowledge/bigdata/phantom/sample-http-proxy/./src/main/resources/external/logback.xml
16:51:47.314 [main] INFO o.t.p.r.i.bootstrap.spring.Bootstrap - Logging framework is now initialized!
16:51:47.423 [main] INFO o.s.c.s.FileSystemXmlApplicationContext - Refreshing org.springframework.context.support.FileSystemXmlApplicationContext@2916117f: startup date [Sun Mar 20 16:51:47 IST 2016]; root of context hierarchy
16:51:47.462 [main] INFO o.s.b.f.xml.XmlBeanDefinitionReader - Loading XML bean definitions from URL [file:/home/kuntal/knowledge/bigdata/phantom/sample-http-proxy/./src/main/resources/external/bootstrap.xml]
16:51:47.609 [main] INFO o.t.p.r.i.c.s.SpringContainerImpl - Initializing Bootstrap extension .... ~~~TrprPlatformCommonBeansContext~~~
16:51:47.613 [main] ERROR o.t.p.r.i.bootstrap.spring.Bootstrap - Fatal error in bootstrap sequence. Cannot continue!
java.lang.NullPointerException: null
at org.trpr.platform.runtime.impl.container.spring.SpringContainerImpl.init(SpringContainerImpl.java:89) ~[runtime-core-1.3.4.jar:na]
at org.trpr.platform.runtime.impl.bootstrap.spring.Bootstrap.start(Bootstrap.java:254) ~[runtime-core-1.3.4.jar:na]
at org.trpr.platform.runtime.impl.bootstrap.spring.Bootstrap.init(Bootstrap.java:136) ~[runtime-core-1.3.4.jar:na]
at org.trpr.platform.runtime.impl.bootstrap.BootstrapLauncher.main(BootstrapLauncher.java:39) [runtime-core-1.3.4.jar:na]
Fatal error in bootstrap sequence. Cannot continue!
java.lang.NullPointerException
at org.trpr.platform.runtime.impl.container.spring.SpringContainerImpl.init(SpringContainerImpl.java:89)
at org.trpr.platform.runtime.impl.bootstrap.spring.Bootstrap.start(Bootstrap.java:254)
at org.trpr.platform.runtime.impl.bootstrap.spring.Bootstrap.init(Bootstrap.java:136)
at org.trpr.platform.runtime.impl.bootstrap.BootstrapLauncher.main(BootstrapLauncher.java:39)
* Trooper runtime shutdown initiated....
* Trooper runtime Stopping....*

In brief ,i couldn't run Phantom by using trooper version <org.trpr.version>1.3.4</org.trpr.version>

Build error at master can't find the org.newsclub:junixsocket:jar

[ERROR] Failed to execute goal on project runtime-oio-uds: Could not resolve dependencies for project com.flipkart.phantom:runtime-oio-uds:jar:1.4.1: Failure to find org.newsclub:junixsocket:jar:1.3 in https://clojars.org/repo was cached in the local repository, resolution will not be reattempted until the update interval of clojars has elapsed or updates are forced -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command

Build failing

Can the jars be published to the clojars repo?

Failed to execute goal on project runtime-oio-uds: Could not resolve dependencies for project com.flipkart.phantom:runtime-oio-uds:jar:1.4.4: Failure to find org.newsclub:junixsocket:jar:1.3 in https://clojars.org/repo was cached in the local repository, resolution will not be reattempted until the update interval of clojars has elapsed or updates are forced

Cluster dashboard shows false open circuit hosts

The cluster dashboard shows a number of host names with open circuits when in fact the circuits are closed on those machines. Issue is with the data stream generated. See
http://<cluster dashboard host>:<port>/turbine.stream.command where:
isCircuitBreakerOpen":false," and still there are host names populated in "openCircuitHostNames" like - openCircuitHostNames":"{w3-web44:1,w3-web92:3,w3-web50:1}"

Link describing motivation for creating Phantom and design overview is down

This link http://tech-blog.flipkart.net/2013/07/proxies-for-resilience-and-fault-tolerance-in-distributed-soa on readme is not working.

Http Headers are not being forwarded in Http proxy

Http headers from the original client request are not being forwarded to the destination server by the Http proxy. This is expected behavior for a transparent protocol proxy.

Feature request: Provide sample including packaging

Hi,

could you please provide (or enhance the existing) sample creating a complete package (tar.gz or zip) for a thrift proxy?
Tried to do it by myself but without look.

Thx,
Michael