netflix / prana

A sidecar for your NetflixOSS based services.

Prana - Making it easy to integrate with NetflixOSS services. Prana exposes Java based client libraries of various services like Eureka, Ribbon, Archaius over HTTP. Prana makes it easy for applications—especially those written in Non-JVM languages—exist in the NetflixOSS eco-system.

Build

We use Gradle for building

./gradlew build

We use the standard Gradle application plugin to build a deployable artifact of Prana

./gradlew distZip

Documentation

Please visit the [wiki] (https://github.com/Netflix/Prana/wiki) for detailed documentation. Please open a GitHub issue if you feel the current documentation is not clear or needs more explanation.

Contributions

Please use the [GitHub Issues] (https://github.com/netflix/Prana/issues) for requests. We actively welcome pull requests.

License

Copyright 2014 Netflix, Inc.

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0

Current state of this project

The current implementation of this project is not used internally at Netflix and therefore updates to the open source have been slow. For more context, see http://ispyker.blogspot.com/2015/10/towards-being-better-about-open-source.html

prana's Issues

Please document resource requirements

I work on an all-Python project. We'd like to run something like Prana as a sidecar with our python apps, but the JVM requirement is scary because of the extra resources it might need. Could you add a resource requirements section to the documentation? If you said how much memory and CPU was needed by a typical real-world Prana installation at Netflix (or as a ratio of memory-per-concurrent-request or whatever) then we'd be able to evaluate whether Prana would work for us without having to tune a JVM just to get started. For context we want it mostly for the Ribbon/Hystrix features; we have other service discovery solutions.

Thanks!

Prana ribbon configuration in documentation?

Hi, I think it would be useful that make some documentation about how to change the default configuration of ribbon.
In ProxyHandler I have found the prefix to modify ribbon config client but nobody mentions it on documentation:

private LoadBalancingHttpClient<ByteBuf, ByteBuf> getClient(String vip) {
        LoadBalancingHttpClient<ByteBuf, ByteBuf> client = httpClients.get(vip);
        if (client == null) {
            IClientConfig config = IClientConfig.Builder.newBuilder("prana_backend").
                    withDefaultValues().
                    withDeploymentContextBasedVipAddresses(vip).
                    build().
                    set(IClientConfigKey.Keys.MaxTotalConnections, 2000).
                    set(IClientConfigKey.Keys.MaxConnectionsPerHost, 2000).
                    set(IClientConfigKey.Keys.OkToRetryOnAllOperations, false).
                    set(IClientConfigKey.Keys.NIWSServerListClassName, DiscoveryEnabledNIWSServerList.class.getName());

            client = RibbonTransport.newHttpClient(new HttpClientPipelineConfigurator<ByteBuf, ByteBuf>(), config);
            httpClients.putIfAbsent(vip, client);

        }
        return client;
    }

As I understand if I want to change something I will need to create properties like this:

prana_backend.ribbon.IsSecure=false
prana_backend.ribbon.ReadTimeout=3000
prana_backend.ribbon.ConnectTimeout=3000
prana_backend.ribbon.MaxAutoRetries=1
prana_backend.ribbon.OkToRetryOnAllOperations=true
prana_backend.ribbon.MaxAutoRetriesNextServer=1
prana_backend.ribbon.FollowRedirects=false
prana_backend.ribbon.ConnIdleEvictTimeMilliSeconds=3600001
prana_backend.ribbon.ServerListRefreshInterval=60001

Thanks!

Return real error instead of a generic "Error forwarding request to origin"

The current error makes it difficult to troubleshoot the issue and "throwable" may contain helpful tips on the reason for the failure:

In the 2 occurrences where ERROR_RESPONSE is being used, here's an improved response:

here

if (Strings.isNullOrEmpty(vip)) {
    serverResponse.getHeaders().set("Content-Type", "application/xml");
    serverResponse.writeString("<status><status_code>500</status_code><message>vip not supplied</message></status>");
    logger.error("VIP not supplied");
    return serverResponse.close();
}

and here

public Observable<Void> call(Throwable throwable) {
    serverResponse.getHeaders().set("Content-Type", "application/xml");
    String msg = "<status><status_code>500</status_code><message>" + throwable.getMessage() + "</message></status>";
    serverResponse.writeString(msg);
    return Observable.just(null);
}

no option to get ip addresses instead of hostname

when i send a curl request to receive the service hosts, i get only hostnames.
curl -X GET http://127.0.0.1:8078/eureka/hosts?appName=s2
when i set the configuration

eureka.instance.preferIpAddress=true

I still get hostnames instead of IP addresses.

In order to get the hostname, i had to change the file HostsHandler.java as follows:
hosts.add(instanceInfo.getIPAddr());

There should be an option to make it configurable

Problem with keepalive connections

It seems, that keepalive connection not handled properly:

$ curl 'http://localhost:8078/dynamicproperties?id=anything' 'http://localhost:8078/healthcheck'; echo
{"anything":null}{}
$ curl 'http://localhost:8078/healthcheck' 'http://localhost:8078/dynamicproperties?id=anything'; echo
<health>ok</health><health>ok</health>

Expected result:

$ curl 'http://localhost:8078/dynamicproperties?id=anything' 'http://localhost:8078/healthcheck'; echo
{"anything":null}<health>ok</health>
$ curl 'http://localhost:8078/healthcheck' 'http://localhost:8078/dynamicproperties?id=anything'; echo
<health>ok</health>{"anything":null}

Netty seems to get stuck on routes with browsers

Using curl you can switch between /ping and /healthcheck and you get expected data. Curl commands don't use http persistent connections.

Using Chrome or Safari, if you go to /ping and then /healthcheck you get pong for healthcheck. Similary if you go to /healthcheck and then /ping you get healthcheck status.

I validated via netstat that Chrome keeps a http persistent connection to Prana. It turns out that as long as this stays open you get stuck in the routes. I closed Chrome to the point of killing the process and was able to see the other url (but then got stuck again).

Prana Sidecar Seems Unusable in Non-AWS Docker Environment

I'm running into an issue where I am using Prana for RabbitMQ service discovery. In the middle, we have our own sidecar for performing the actual healthcheck, as well as metric reporting.

In AWS it looks like this:

This works in AWS, because it pulls the hostname from the AWS Metadata API:

$ curl -v 'http://prana/eureka/hosts?appName=RABBIT'
["ec2-55-66-111-77.us-west-2.compute.amazonaws.com"]

However in a non-AWS Docker container:

• where the Prana IP is different from the service it is a sidecar to
• and eureka.validateInstanceId=false

it uses Prana's IP address, like so:

Due to this, it does not seem possible to use Prana + Eureka in a non-AWS development environment.

Load Groovy based request handlers during startup of Prana from filesystem

Prana serves the purpose of exposing features of the various client libraries like Eureka, Servo, Ribbon etc. Users should be able to expose APIs of those libraries in a manner that's suitable to their application in case we don't or our default APIs don't serve their use case.

Users of Prana could pass a command line argument --filters-location=/path/to/filters. Prana would scan the location for Groovy classes which implements the PranaHttpExtension interface and compile them and wire them up with the http router. Extensions would be only loaded during the application startup.

The Prana interface looks like -

public interface PranaHttpExtension extends RequestHandler<ByteBuf, ByteBuf> {
    public String getRoute();
}

and a sample extension implementation would look like this

public class ExampleHttpHandler implements PranaHttpExtension {

    private DiscoveryClient discoveryClient;

    private ObjectMapper objectMapper;

    @Inject
    public ExampleHttpHandler(DiscoveryClient discoveryClient, ObjectMapper objectMapper) {
        this.discoveryClient = discoveryClient;
        this.objectMapper = objectMapper;
    }

    @Override
    public String getRoute() {
        return "/getregions";
    }


    @Override
    public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) {
        response.getHeaders().add("Content-Type", "application/json");
        try {
            byte[] bytes = objectMapper.writeValueAsBytes(discoveryClient.getAllKnownRegions());
            response.writeBytes(bytes);
            response.setStatus(HttpResponseStatus.OK);
        } catch (JsonProcessingException e) {
            response.setStatus(HttpResponseStatus.INTERNAL_SERVER_ERROR);
        }
        return response.close();
    }
}

Prana extensions would be able to inject objects that are instantiated by Guice.

Registered healthcheck, status, and homepage URLs use application's port, not Prana's port

Using the following configuration:

prana.host.healthcheck.url=http://127.0.0.1:5000/hc
prana.host.healthcheck.timeout=1500
prana.proxy.req.acceptencoding=gzip

archaius.deployment.applicationId=EXAMPLEAPP
archaius.deployment.environment=prod

eureka.name=EXAMPLEAPP
eureka.vipAddress=EXAMPLEAPP
eureka.port=5000
eureka.environment=prod
eureka.appinfo.initial.replicate.time=1
eureka.preferSameZone=true
eureka.region=us-west-2
eureka.datacenter=cloud
eureka.shouldUseDns=true
eureka.eurekaServer.domainName=eureka.example.net
eureka.eurekaServer.port=8080
eureka.eurekaServer.context=eureka/v2

and starting Prana with:

./Prana -c ./config -p 8080

In Eureka I see:

<application>
  <name>RABBIT3</name>
  <instance>
    <hostName>ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com</hostName>
    <app>RABBIT3</app>
    <ipAddr>172.17.0.112</ipAddr>
    <status>UP</status>
    <overriddenstatus>UNKNOWN</overriddenstatus>
    <port enabled="true">5000</port>
    <securePort enabled="false">443</securePort>
    <countryId>1</countryId>
    <dataCenterInfo class="com.netflix.appinfo.AmazonInfo">
      <name>Amazon</name>
      <metadata>
        <availability-zone>us-west-2a</availability-zone>
        <public-ipv4>xx.xx.xx.xx</public-ipv4>
        <instance-id>i-xxxxxxxx</instance-id>
        <public-hostname>ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com</public-hostname>
        <local-ipv4>172.31.30.174</local-ipv4>
        <ami-id>ami-xxxxxxxx</ami-id>
        <instance-type>r3.8xlarge</instance-type>
      </metadata>
    </dataCenterInfo>
    <leaseInfo>
      <renewalIntervalInSecs>30</renewalIntervalInSecs>
      <durationInSecs>90</durationInSecs>
      <registrationTimestamp>1429647009479</registrationTimestamp>
      <lastRenewalTimestamp>1429647009479</lastRenewalTimestamp>
      <evictionTimestamp>0</evictionTimestamp>
      <serviceUpTimestamp>1429647009479</serviceUpTimestamp>
    </leaseInfo>
    <metadata class="java.util.Collections$EmptyMap"/>
    <appGroupName>UNKNOWN</appGroupName>
    <homePageUrl>http://ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com:5000/</homePageUrl>
    <statusPageUrl>http://ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com:5000/Status</statusPageUrl>
    <healthCheckUrl>http://ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com:5000/healthcheck</healthCheckUrl>
    <vipAddress>rabbit3</vipAddress>
    <isCoordinatingDiscoveryServer>false</isCoordinatingDiscoveryServer>
    <lastUpdatedTimestamp>1429647009479</lastUpdatedTimestamp>
    <lastDirtyTimestamp>1429647009092</lastDirtyTimestamp>
    <actionType>ADDED</actionType>
  </instance>
</application>

Note specifically the homepage, status, and health checks are referring to port 5000:

    <homePageUrl>http://ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com:5000/</homePageUrl>
    <statusPageUrl>http://ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com:5000/Status</statusPageUrl>
    <healthCheckUrl>http://ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com:5000/healthcheck</healthCheckUrl>

Meanwhile in the logs, I'm seeing the Prana bind to port 8080:

3134 [main] INFO io.reactivex.netty.server.AbstractServer - Rx server started at port: 8080

Docker image for Prana

Is there any docker image for prana side car?

how to configure for a eureka cluster? DNS based

I've got a DNS based Eureka Cluster which has 3 nodes. I need to point Prana to the dns domain but the same options for Eureka don't seem to work.

I'm using:

eureka.datacenter=cloud
eureka.registerWithEureka=True
eureka.fetchRegistry=True
eureka.useDnsForFetchingServiceUrls=True
eureka.eurekaServerDNSName=dev.example.com
eureka.eurekaServerPort=8080
eureka.eurekaServerURLContext=eureka/v2
eureka.region=us-west-2

Any help/examples would be helpful!

how do i configure prana to expose the hystrix.stream

Hi,

what do i need to configure in the prana sidecar in order to expose the hystrix.stream?
my goal is to see the circuit breaker in real time in the hystrix dashboard

Thanks,

Uzi

Wrong URL's published in Eureka

By default Prana registers:

http://hostname:8080/Status
http://hostname:8080/healthcheck

These URL's aren't right and should instead match either the Prana config'ed port or the Karyon admin console port.

Prana service breaks Eureka console

I just downloaded the Prana zip file and tried to fire it up. Perhaps this should be a separate issue, but no matter what command line arguments I provided, it wouldn't use my external config file and kept defaulting to the config file bundled in the Prana-0.0.1.jar file. After unzipping the jar, editing the config to point to my Eureka server, then zipping it back up and replacing the jar, Prana successfully registered my service. Going to http://localhost:8761/eureka/apps, I see all of my registered apps, however if I go to the dashboard at http://localhost:8761/, it get a pretty nasty error. As soon as I take down my Prana service, the Eureka dashboard displays properly again. I'm not sure if this is an issue with Prana or Eureka, but I don't see this problem with any of the spring cloud services I've developed. Here is the error:

FreeMarker template error (DEBUG mode; use RETHROW in production!): The following has evaluated to null or missing: ==> amiCount.key [in template "eureka/status.ftl" at line 35, column 26] ---- Tip: It's the step after the last dot that caused this error, not those before it. ---- Tip: If the failing expression is known to be legally refer to something that's null or missing, either specify a default value like myOptionalVar!myDefault, or use <#if myOptionalVar??>when-present<#else>when-missing. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)?? ---- ---- FTL stack trace ("~" means nesting-related): - Failed at: ${amiCount.key} [in template "eureka/status.ftl" at line 35, column 24] ---- Java stack trace (for programmers): ---- freemarker.core.InvalidReferenceException: [... Exception message was already printed; see it above ...] at freemarker.core.InvalidReferenceException.getInstance(InvalidReferenceException.java:116) at freemarker.core.EvalUtil.coerceModelToString(EvalUtil.java:346) at freemarker.core.Expression.evalAndCoerceToString(Expression.java:82) at freemarker.core.DollarVariable.accept(DollarVariable.java:40) at freemarker.core.Environment.visit(Environment.java:312) at freemarker.core.MixedContent.accept(MixedContent.java:62) at freemarker.core.Environment.visitByHiddingParent(Environment.java:333) at freemarker.core.IteratorBlock$Context.runLoop(IteratorBlock.java:159) at freemarker.core.Environment.visitIteratorBlock(Environment.java:559) at freemarker.core.IteratorBlock.accept(IteratorBlock.java:67) at freemarker.core.Environment.visit(Environment.java:312) at freemarker.core.MixedContent.accept(MixedContent.java:62) at freemarker.core.Environment.visitByHiddingParent(Environment.java:333) at freemarker.core.IteratorBlock$Context.runLoop(IteratorBlock.java:159) at freemarker.core.Environment.visitIteratorBlock(Environment.java:559) at freemarker.core.IteratorBlock.accept(IteratorBlock.java:67) at freemarker.core.Environment.visitByHiddingParent(Environment.java:333) at freemarker.core.IfBlock.accept(IfBlock.java:48) at freemarker.core.Environment.visit(Environment.java:312) at freemarker.core.MixedContent.accept(MixedContent.java:62) at freemarker.core.Environment.visit(Environment.java:312) at freemarker.core.Environment.process(Environment.java:290) at freemarker.template.Template.process(Template.java:312) at org.springframework.web.servlet.view.freemarker.FreeMarkerView.processTemplate(FreeMarkerView.java:367) at org.springframework.web.servlet.view.freemarker.FreeMarkerView.doRender(FreeMarkerView.java:284) at org.springframework.web.servlet.view.freemarker.FreeMarkerView.renderMergedTemplateModel(FreeMarkerView.java:234) at org.springframework.web.servlet.view.AbstractTemplateView.renderMergedOutputModel(AbstractTemplateView.java:167) at org.springframework.web.servlet.view.AbstractView.render(AbstractView.java:303) at org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1244) at org.springframework.web.servlet.DispatcherServlet.processDispatchResult(DispatcherServlet.java:1027) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:971) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:967) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:858) at javax.servlet.http.HttpServlet.service(HttpServlet.java:622) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:843) at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextHeaderFilter.doFilterInternal(EndpointWebMvcAutoConfiguration.java:295) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:102) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:85) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:68) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:518) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1091) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:668) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1521) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1478) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745)
Whitelabel Error Page

This application has no explicit mapping for /error, so you are seeing this as a fallback.
Mon Sep 21 13:53:03 EDT 2015
There was an unexpected error (type=OK, status=200).
The following has evaluated to null or missing: ==> amiCount.key [in template "eureka/status.ftl" at line 35, column 26] ---- Tip: It's the step after the last dot that caused this error, not those before it. ---- Tip: If the failing expression is known to be legally refer to something that's null or missing, either specify a default value like myOptionalVar!myDefault, or use <#if myOptionalVar??>when-present<#else>when-missing</#if>. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)?? ---- ---- FTL stack trace ("~" means nesting-related): - Failed at: ${amiCount.key} [in template "eureka/status.ftl" at line 35, column 24] ----