nginx-clojure / nginx-clojure

Nginx module for embedding Clojure or Java or Groovy programs, typically those Ring based handlers.

Nginx-Clojure

Nginx-Clojure is a Nginx module for embedding Clojure or Java or Groovy programs, typically those Ring based handlers.

Core Features

The latest release is v0.6.0, more detail changes about it can be found from Release History.

1. Compatible with Ring and obviously supports those Ring based frameworks, such as Compojure etc.
2. Http Services by using Clojure / Java / Groovy to write simple handlers for http services.
3. Nginx Access Handler by Clojure / Java / Groovy
4. Nginx Header Filter by Clojure / Java / Groovy
5. Nginx Body Filter by Clojure / Java / Groovy
6. Nginx Log Handler by Clojure / Java / Groovy
7. HTTP V2 support in both standard edition and embedded edition which are compiled against Nginx 1.18.0+
8. Support Java 8, 9, 10, 11, 12, 19
9. Support to use jdk19 built-in coroutine viz. Continuation
10. Pub/Sub Among Nginx Worker Processes
11. Shared Map based on shared memory & Shared Map based Ring session store
12. Support Sente, see this PR
13. Support Per-message Compression Extensions (PMCEs) for WebSocket
14. APIs for Embedding Nginx-Clojure into a Standard Clojure/Java/Groovy App
15. Server Side Websocket
16. A build-in Jersey container to support java standard RESTful web services (JAX-RS 2.0)
17. Tomcat 8 embedding support (so servlet 3.1/jsp/sendfile/JSR-356 websocket work within nginx!)
18. Dynamic proxying by using Clojure / Java / Groovy to write a simple nginx rewrite handler to set var or return errors before proxy pass or content ring handler
19. Non-blocking coroutine based socket which is Compatible with Java Socket API and works well with largely existing java library such as apache http client, mysql jdbc drivers. With this feature one java main thread can handle thousands of connections.
20. Handle multiple sockets parallel in sub coroutines, e.g. we can invoke two remote services at the same time.
21. Asynchronous callback API of socket/Channel for some advanced usage
22. Long Polling & Server Sent Events
23. Run initialization clojure code when nginx worker starting
24. Support user defined http request method
25. Compatible with the Nginx lastest most mainline version 1.23.3. (Nginx 1.22.X, 1.20.X, 1.18.x, 1.14.x, 1.12.x, 1.8.x, 1.6.x, 1.4.x is also ok, older version is not tested and maybe works.)
26. One of benifits of Nginx is worker processes are automatically restarted by a master process if they crash
27. Utilize lazy headers and direct memory operation between Nginx and JVM to fast handle dynamic contents from Clojure or Java code.
28. Utilize Nginx zero copy file sending mechanism to fast handle static contents controlled by Clojure or Java code.
29. Support Linux x64, Linux x86 32bit, Win32, Win64 and Mac OS X. Freebsd version can also be got from Freebsd ports.

By the way it is very fast, the benchmarks can be found HERE(with wrk2).

Jar Repository

Nginx-Clojure has already been published to https://clojars.org/ whose maven repository is

<repository>
  <id>clojars.org</id>
  <url>http://clojars.org/repo</url>
</repository>

After adding clojars repository, you can reference nginx-clojure 0.6.0 , e.g.

Leiningen (clojure, no need to add clojars repository which is a default repository for Leiningen)

[nginx-clojure "0.6.0"]

Gradle (groovy/java)

compile "nginx-clojure:nginx-clojure:0.6.0"

Maven

<dependency>
  <groupId>nginx-clojure</groupId>
  <artifactId>nginx-clojure</artifactId>
  <version>0.6.0</version>
</dependency>

Documents

• Quick Start
• Downloads
• Installation
• Configuration
  • JVM Path,Class Path & Other JVM Options
  • Initialization Handler for nginx worker
  • Content Ring Handler for Location
  • Coroutine/Asynchronous Client Channel/Thread Pool
• Nginx Handlers
  • Nginx Rewrite Handler
  • Nginx Access Handler
  • Nginx Header Filter
  • Nginx Body Filter
  • Nginx Log Handler
• Advanced Topic
  • Embedding Nginx-Clojure into a standard App
  • Server Channel for Long Polling & Server Sent Events
  • Pub/Sub Among Nginx Worker Processes
  • Shared Map & Session Store
  • Asynchronous Client Channel
  • About Logging
  • Sever Side WebSocket
  • Java standard RESTful web services with Jersey
  • Embedding Tomcat
  • Use Spring Framework with Nginx-Clojure Java Handlers
  • Example Project about Jersey & Spring with Nginx-Clojure
  • More about Nginx Worker Process
  • More about Nginx-Clojure
• Directives Reference
• API Reference (Clojure)
• Example Projects
• Github Repository of Documents

License

Copyright © 2013-2023 Zhang, Yuexiang (xfeep) and released under the BSD 3-Clause license.

This program uses:

• Re-rooted ASM bytecode engineering library which is distributed under the BSD 3-Clause license
• Modified Continuations Library written by Matthias Mann is distributed under the BSD 3-Clause license

nginx-clojure's Issues

Running tests

What's the proper way of running tests for this project?
I tried to run it with:

$ lein with-profile unittest test

1. On a run, where I use unchanged source code, I get:

...
Exception in thread "main" java.lang.NoClassDefFoundError: nginx/clojure/SuspendExecution, compiling:(clj_http/conn_mgr.clj:26:26)
...

When I remove jvm-opts "-javaagent:target/nginx-clojure-0.2.4.jar=mb" and add "src/java" and "src/clojure" to :java-source-paths and :source-paths, respectively I get the tests to compile, but I get this errors:

1. test_all.clj expects that the Nginx is up and running on localhost, but I don't know where to find configuration with which nginx should run
2. When I run all the other tests without test_all.clj, they all fail. They all have similar stack trace:

...
Testcase: testUninitialized(nginx.clojure.UninitializedTest): Caused an ERROR
null
java.lang.NullPointerException
at nginx.clojure.Stack.(Stack.java:83)
at nginx.clojure.Coroutine.(Coroutine.java:120)
at nginx.clojure.Coroutine.(Coroutine.java:105)
at nginx.clojure.UninitializedTest.testUninitialized(UninitializedTest.java:23)
...

This is not surprising, since the Stack constructor is accessing the static variable which wasn't initialised.

Verifying option for auto generated waving configurations needed by coroutine based socket

Supports verifying option for auto-generated waving configurations needed by coroutine based socket.

e.g.

jvm_options "-javaagent:jars/nginx-clojure-0.2.1.jar=vmb";

The v means enabling verifying option then if some method need be waved but there 's no right waving configurations nginx-clojure will report this problem. Then user can add the additional information to the waving configurations or redo something to get the auto generated waving configurations again.

Run initialization clojure code when nginx worker starting.

This feature is part of issue #5 ngx shared dic.
We 'll provide this feature in the coming release 0.2.0.
You can embed clojure code in the http { block to do initialization when nginx worker starting. e.g

http {
......
    clojure;
    clojure_code '
    (fn[ctx]
        (.println System/err "on http clojure context")
        )
    ';
....
}

Because the maybe more than one nginx worker process, so this code will run everytime per worker starting. If you use SharedHashMap to share data among nginx worker processes, Java file lock can be used to let only one nginx worker process do the initialization.

built-in jvm variable #{pno} doesn't work

This bug was introduced by commit 31cbc1a which will do evaluation twice ( 1. check length at postconfiguration, 2. check length at worker process initialization).

e.g

worker_processes  8;

http {

    jvm_options "-Xdebug";
    jvm_options "-Xrunjdwp:server=y,transport=dt_socket,address=240#{pno},suspend=n";

}

nginx can not start because all nginx workers try to bind port 240.

It looks promising! a "thank you" and a naive question

So nginx-clojure is not working in a normal reverse-proxy mode, deployment-wise ? By "reverse-proxy mode" I mean usually we deploy 2 servers, one is nginx server, one is a java web server reverse-proxied(via http or name-pipe) by the front nginx.

With nginx-clojure, when I do deployment, I only need to config nginx by providing my clojure code(and JVM configuration of course). Then I start my nginx and I'll be all set. Is this understand correct ? If so, I like this light-weight deployment mode.

I love using nginx and clojure, although clojure is still on my studying stage.

On Windows a little many write events happen and these events seem useless.

Steps for reproduce this issue

1. On Windows

2. Enable coroutine based socket support

3. Turn on debug of coroutine based socket

   jvm_options "-Dnginx.clojure.logger.socket.level=debug";

4. Use apache http client & clj-http , e.g.

   (let [rt (client/get 
               "http://mirror.bit.edu.cn/apache/httpcomponents/httpclient/RELEASE_NOTES-4.3.x.txt")]
     (print (:body rt)))

All things go well but in the error.log we can see a little many write events happened and they seems useless and they didn't cause any real writing operation. The debug info just like

2014-08-08 17:10:45[debug]:socket#12526832: find suspend on YIELD_CONNECT, we'ill resume it
2014-08-08 17:10:45[debug]:socket#12526832: enter write offset 0 len 202
2014-08-08 17:10:45[debug]:socket#12526832: write offset 0 len 202 return 202, total 202
2014-08-08 17:10:45[debug]:socket#12526832: enter read offset 0 len 8192
2014-08-08 17:10:45[debug]:socket#12526832: read offset 0 len 8192 return -27, total 0
2014-08-08 17:10:45[debug]:socket#12526832: yield read
...  yield debug trace omited here
2014-08-08 17:10:45[debug]:socket#12526832: on write status=0
2014-08-08 17:10:45[debug]:socket#12526832: on write status=0
2014-08-08 17:10:45[debug]:socket#12526832: on write status=0
2014-08-08 17:10:45[debug]:socket#12526832: on write status=0
2014-08-08 17:10:45[debug]:socket#12526832: on write status=0
2014-08-08 17:10:45[debug]:socket#12526832: on write status=0
2014-08-08 17:10:45[debug]:socket#12526832: on write status=0
2014-08-08 17:10:45[debug]:socket#12526832: on write status=0
2014-08-08 17:10:45[debug]:socket#12526832: on read status=0
2014-08-08 17:10:45[debug]:socket#12526832: find suspend on YIELD_READ, we'ill resume it
2014-08-08 17:10:45[debug]:socket#12526832: read offset 0 len 8192 return 1436, total 1436
2014-08-08 17:10:45[debug]:socket#12526832: read offset 1436 len 6756 return -27, total 1436
2014-08-08 17:10:45[debug]:socket#12526832: enter read offset 0 len 1024
2014-08-08 17:10:45[debug]:socket#12526832: read offset 0 len 1024 return 1024, total 1024
2014-08-08 17:10:45[debug]:socket#12526832: enter read offset 0 len 1024
...

These useless write events happened after sending the http request and no read data reached yet.
It's because

1. On windows nginx uses select event model so if the write event isn't handled by write operation the event will be triggered again and again.
2. The connection may be keep alive and the server can still receive data even if one http request completed.
3. But apache http client won't handle this write event, viz. it won't write any more data because it have sent the whole request to the server.
4. So this unhandled write event will be triggered again and again until the socket being closed.

Although these events seem harmless and won't stop the next read event but they waste some cpu cycles. Optimization method can be done to resolve this issue.

Should Clone ThreadLocals for Coroutines

Vars defined with metadata mark them as dynamic are implemented by Java ThreadLocals so there is a different copy for every thread.
When under couroutine enabled mode one thread can run several coroutines at the same time , something will be wrong for these coroutines share one copy of ThreadLocal of the thread. e.g.

(def ^:dynamic *mybinding* "")

(def a 
  (Coroutine. 
    (fn []
        (binding [*mybinding* "ca"]
          (println "1 in a:" *mybinding*)
          (Coroutine/yield)
          (println "2 in a:" *mybinding*)
      )))

 (def b
     (Coroutine. 
       (fn []
           (binding [*mybinding* "cb"]
             (println "1 in b:" *mybinding*)
             (Coroutine/yield)
             (println "2 in b:" *mybinding*)
         ))))

(.resume a)
(.resume b)
(.resume a)
(.resume b)

The right result should be

1 in a: ca
1 in b: cb
2 in a: ca
2 in b: cb

Nginx won't start successfully

This is on Amazon Linux. I tried the binaries and compiling from source and I get the same error both times (with nginx-clojure-0.2.5):

Exception in thread "main" java.lang.NoClassDefFoundError: nginx/clojure/MiniConstants
Caused by: java.lang.ClassNotFoundException: nginx.clojure.MiniConstants
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
2014/09/26 14:03:39 [error] 11564#0: can not initialize jvm memory util
2014/09/26 14:03:39 [error] 11564#0: jvm start times 1
2014/09/26 14:03:39 [alert] 11430#0: worker process 11564 exited with fatal code 2 and cannot be respawned

Supports reference variables in jvm_options & different jvm debug ports for jvm processes

Supports reference variables in jvm_options , e.g

#define nginx_clojure_jar
jvm_var nginx_clojure_jar '/home/who/git/nginx-clojure/target/nginx-clojure-0.2.5.jar';
#reference variable nginx_clojure_jar, starts with #{ different from nginx variable
jvm_options "-javaagent:#{nginx_clojure_jar}=mb";
jvm_options "-Xbootclasspath/a:#{nginx_clojure_jar}:jars/clojure-1.5.1.jar"; 

Supports different jvm debug ports for jvm processes by using built-in variable pno which is a nginx worker process number which starts from 1 to n. e.g.

worker_processes  8;

http {

    jvm_options "-Xdebug";
    jvm_options "-Xrunjdwp:server=y,transport=dt_socket,address=240#{pno},suspend=n";

}

Then 8 jvm instances embed into 8 nginx work processes will get different debug ports from 2401 ~ 2408. We can use Java IDE or jdb to connect them and debug the java code.

Supports auto turn on thread pool mode when turning on Run Tool Mode

When turns on run tool mode to generate coroutine waving configuration file, the nginx worker processes need to set to 1. But sometimes we need to access services provide by the same nginx instance, e.g. proxyed external http service, it will make worker deadly blocked. So we should auto turn on thread pool mode with turning on Run Tool Mode to avoid this problem.

Support to close coroutine based socket from non-main thread

Some libraries want to reuse socekt and so use another thread to monitor socket keepalive timeout event or error event, such as Java build-in HttpURLConnection implementation. If the monitor find socket must be closed it will close it directly in non-main thread then will cause some exception just like

Exception in thread "Keep-Alive-Timer" java.lang.IllegalAccessError: close method of coroutine based sockets can only be called in main thread
    at nginx.clojure.net.NginxClojureSocketImpl.close(NginxClojureSocketImpl.java:272)
    at java.net.Socket.close(Socket.java:1437)
    at sun.net.www.http.HttpClient.closeServer(HttpClient.java:970)
    at sun.net.www.http.KeepAliveCache.run(KeepAliveCache.java:199)
    at java.lang.Thread.run(Thread.java:744)

So to be friendly with this existing libraries we should support close coroutine based socket from non-main thread, typically we log warning messages and post an event by pipe to main thread to let it handle socket close event safely.

asynchronous callback API of socket for some advanced usage

Support asynchronous callback API of socket for some advanced usage.
The callback handler can be either Java Object or Clojure Function.
eg.

(defn http-get-handler [socket, type, status]
  (case type 
    "connect" (handle-connect-event ...)
     "read"  (handle-read-event ...)
     "write" (handle-read-event ...)
     "release" (handle-release-event ...)
))

public interface NginxClojureSocketHandler {
    public void onConnect(socket,  status);
    public void onRead(socket,  status);
    public void onWrite(socket,  status);
    public void onRelease(socket,  status);
}

Nginx worker crashes when to fetch http header "authorization" from request

Here 's a simple example to reproduce this issue.

(fn [req]
 (log/info "authorization=" (get-in req [:headers "authorization"])))

Nginx worker will crash and be created again.

Thanks to Michal Pisanko who helped me to find this bug.

Nginx Clojure completely hogging worker_processes as long as request remains active

We're developing on a setup where we have a central nginx setup containing both regular proxy passes as well as the nginx-clojure configuration.

This nginx-clojure has a groovy handler which talks to marathon (which itself is also managed by this nginx instance) and then dynamically determines the proxy pass.

At some point we noticed the Nginx process completely hanging on us when we generated some load. We started debugging and found out that when we set 'worker_processes 1' we could instantly hang nginx.

The path the request takes is:
browser -> nginx [clojure-logic] -> marathon-server-url -> nginx [basic-proxy-pass] -> marathon server -> nginx [basic-proxy-pass] -> mesos-server

The log indicates that it tries to connect to the marathon-server-url, but is blocked by the nginx-basic-proxy-pass. When we set 'worker_processes 4' the logic works as long as there aren't more than 4 requests browser -> nginx-clojure-logic.

daemon off;

worker_processes 1;
pid /run/nginx.pid;

user root;

events {
  worker_connections 768;
}

http {

  sendfile on;
  tcp_nopush on;
  tcp_nodelay on;
  keepalive_timeout 65;
  types_hash_max_size 2048;
  include /opt/nginx/conf/mime.types;
  default_type application/octet-stream;
  gzip on;
  gzip_disable "msie6";

  include /opt/nginx/conf.d/*.conf;

  include       mime.types;

  jvm_path "/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so";

  ### begin nginx-marathon config 
  jvm_options "-DNGINX_WORKER_PID=#{pno}";

  # j_path points to the 'jars' directory within the nginx installation
  jvm_var j_path 'jars';
  jvm_var base_jars '#{j_path}/nginx-clojure-0.2.6.jar';

  jvm_var nm_path '#{j_path}/nginx-marathon';
  jvm_var nginx_marathon_depjars '#{nm_path}/lib/groovy-2.3.7.jar:#{nm_path}/lib/logback-classic-1.1.2.jar:#{nm_path}/lib/logback-core-1.1.2.jar:#{nm_path}/lib/slf4j-api-1.7.6.jar:#{nm_path}/lib/marathon-client-@[email protected]:#{nm_path}/lib/dagger-1.1.0.jar:#{nm_path}/lib/javax.inject-1.jar:#{nm_path}/lib/feign-core-6.1.1.jar:#{nm_path}/lib/gson-2.2.4.jar:#{nm_path}/lib/feign-gson-6.1.1.jar';
  jvm_var nginx_marathon_jars '#{nm_path}/@projectName@-@[email protected]:#{nginx_marathon_depjars}';

  ### end nginx-marathon config

  jvm_options "-Djava.class.path=#{base_jars}:#{nginx_marathon_jars}";

  server {
    listen 80 default_server;
    server_name _;

    client_max_body_size 200M;

    set $nginx_marathon_proxy_pass_host "undefined";
    location / {
      resolver                 10.10.1.66;  
      handler_type             'groovy';

      rewrite_handler_name     'com.mycompany.nginx.marathon.NginxMarathonServiceDiscoveryHandler';
      proxy_pass               $nginx_marathon_proxy_pass_host;

      proxy_redirect           off;
      proxy_read_timeout       2m;

      proxy_set_header         Host            $host;
      proxy_set_header         X-Real-IP       $remote_addr;
      proxy_set_header         X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_max_temp_file_size 0;
    }
  }
}

How to pass context from initialization code to ring handler

Hi, I'd like to use nginx-clojure as a gateway into an internal SOA based on ZooKeeper. To do this, in the initialization code I must initialize the ZooKeeper connection. Then, the ring handler uses this connection to proxy requests appropriately.

Is it possible to get the return value of the initialization code in the ring request map?

If not, I think it would be good to include it at the key :nginx-init.

Supports 64-bit JDK on 64-bit Windows

From the next release official binary releases will include win64 version. With it we can use 64-bit JDK on 64-bit Windows.

Problems with HTTP Redirect 302

Hi,

i'm trying to use nginx-clojure to make work a very simple application with ring&compojure. I've experienced a problem when the response is a redirect:

(defn root []
  (response/redirect "/path/to/redirect"))

If I use the ring server the request is made properly and a 302 code is received with the correct Location header. When I try to use this app with the nginx-clojure module I get something similar to the next:

[~]$ curl -k -v -X GET https://127.0.0.1:8000/foo                                                                                                [1.9.3-p327] 
* About to connect() to 127.0.0.1 port 8000 (#0)
*   Trying 127.0.0.1...
* Adding handle: conn: 0x10c51a0
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x10c51a0) send_pipe: 1, recv_pipe: 0
* Connected to 127.0.0.1 (127.0.0.1) port 8000 (#0)
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using ECDHE-RSA-AES256-GCM-SHA384
* Server certificate:
*        subject: C=ES; ST=Barcelona; L=Barcelona; O=Internet Widgits Pty Ltd; [email protected]
*        start date: 2014-01-25 23:55:24 GMT
*        expire date: 2015-01-25 23:55:24 GMT
*        issuer: C=ES; ST=Barcelona; L=Barcelona; O=Internet Widgits Pty Ltd; [email protected]
*        SSL certificate verify result: self signed certificate (18), continuing anyway.
> GET /foo HTTP/1.1
> User-Agent: curl/7.32.0
> Host: 127.0.0.1:8000
> Accept: */*
> 
* SSLv3, TLS alert, Client hello (1):
* Empty reply from server
* Connection #0 to host 127.0.0.1 left intact
curl: (52) Empty reply from server

As you see, I receive a Empty reply from server, a 204 status code. My nginx configuration is the next, i've enabled SSL:

daemon  off;
master_process  off;
worker_processes  8;
error_log   logs/error.log;

events {
    worker_connections  40;
    #debug_connection 127.0.0.1;
}


http {

    include       mime.types;
    default_type  application/octet-stream;

    sendfile        on;
    keepalive_timeout  65;

    jvm_path "/usr/lib/jvm/java-7-openjdk/jre/lib/amd64/server/libjvm.so";
    jvm_options "-Djava.class.path=/home/jon/.m2/repository/org/clojure/clojure/1.5.1/clojure-1.5.1.jar:/home/jon/src/nginx-1.4.4/nginx-clojure/target/nginx-clojure-0.1.1.jar:/home/jon/projects/lastfm-four/target/:/home/jon/projects/lastfm-four/src/:/home/jon/projects/lastfm-four/target/lastfm-four-0.1-standalone.jar";

    jvm_options "-Xdebug";
    jvm_options "-Xrunjdwp:server=y,transport=dt_socket,address=8400,suspend=n";

    server {
        listen       8000;
        server_name  localhost;

        ssl on;
        ssl_certificate /home/jon/.ssl/server.crt;
        ssl_certificate_key /home/jon/.ssl/server.key;

        location / {
            clojure;
            clojure_code '
            (do
              (require \'[lastfm-four.handler :as handler]) 
                handler/app)
            ';
        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }

}

Do you know if this is a problem with my nginx configuration? A problem with nginx-clojure?

Thank you. Cheers.

rewrite_handler_name does not work without content handler

rewrite_handler_name & rewrite_handler_code don't work without handler_code or handler_name

This example does not work

        set $destination "should_get_overwritten";
        location / {
           resolver 8.8.8.8;
           handler_type 'java';
           rewrite_handler_name 'my.ProxyHandler';
           proxy_pass $destination;
        }

But old directive clojure & clojure_rewrite_code works. e.g. This example works

        set $destination "should_get_overwritten";
        location / {
           resolver 8.8.8.8;
           clojure;
           clojure_rewrite_code '
           (do 
                      (require \'[my.clj-handler :as m])
                      m/clj-proxy-handler)';        
           proxy_pass $destination;
        }

Thanks Eric Kubacki for finding this bug.

Consider publishing to Maven Central and/or JCenter

Great module, thanks very much for making it available.

Can you please consider publishing your module to:

• Bintray jcenter (https://bintray.com/docs/uploads/uploads_includingyourpackagesinjcenter.html, also supports pushing through to Maven Central)
• Maven Central (http://maven.apache.org/guides/mini/guide-central-repository-upload.html)

This so it can be easily used from clojure/groovy/java code that makes use of nginx-clojure, like the nginx-marathon project we're currently implementing with my team.
Otherwise I have to store the jar in my code repo, and I rather not. I've already got it for 0.2.6, but would rather not have any more 😄

Thanks

provide a tool to make setting of coroutine based socket easier

Provide a tool to make setting of coroutine based socket easier.
This tool can auto generate class waving configurations.
e.g. we want to generate class waving configurations about mysql jdbc driver, we just use this tool to run all of the mysql jdbc driver unit test cases. After all test cases are done, class waving configuration file will be generated.

worker dead (infinite loop?) after enabling coroutine

centos 6.5 (virtualbox vm), nginx-clojure module on openresty 1.5.12, sun jdk 1.7.0_55. After enabling the coroutine, the request to /clojure never returns and one of the two workers consumes one cpu core completely (100% cpu usage, the vm box has two cpu cores), looks like an infinite loop somewhere in the embedded jvm. It is all good before the coroutine is enabled.

===== nginx.conf =====
worker_processes auto;
events {
worker_connections 1024;
}

http {
jvm_path "/opt/jdk1.7.0_55/jre/lib/amd64/server/libjvm.so";
jvm_options "-Djava.class.path=jars:jars/nginx-clojure-0.2.0.jar:jars/clojure-1.5.1.jar:jars/test-0.1.0-SNAPSHOT-standalone.jar";
jvm_options "-Xms1024m";
jvm_options "-Xmx1024m";
jvm_options "-javaagent:jars/nginx-clojure-0.2.0.jar=mb";
jvm_options "-Xbootclasspath/a:jars/nginx-clojure-0.2.0.jar:jars/clojure-1.5.1.jar";
jvm_options "-Dnginx.clojure.wave.udfs=my-wave-cfg.txt";
#jvm_options "-Dnginx.clojure.wave.CfgToolOutFile=/tmp/my-wave-cfg.txt";

include       mime.types;
default_type  application/octet-stream;

sendfile        on;
keepalive_timeout  65;

server {
    listen       80;
    server_name  localhost;

    location /clojure {
      clojure;
      clojure_code ' 
        (do
          (require \'[test.core :as tcore])
          tcore/foo)
      ';
   }
    location /dump {
     clojure;
     clojure_code ' 
           (do (import \'[nginx.clojure.wave SuspendMethodTracer]) 
             (fn[req]
                (SuspendMethodTracer/dump) 
                {:status 200, :body "ok", :headers {"content-type"  "text/plain"}} 
             ))
      ';       

   }

    location / {
        root   html;
        index  index.html index.htm;
    }

    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   html;
    }
}

}
==== nginx.conf ====

==== ring handler ====
(ns test.core
(:require [clj-http.client :as client]))

(defn foo
"I don't do a whole lot."
[req]
(client/get "http://www.google.com/"))
==== ring handler ====

The first registered handler will not work if there 's a asynchronous reading of request body

The first registered handler will not work if there 's a asynchronous reading of request body. This bug is only exists on version 0.2.7.

We can simple add a nothing-to-do nginx worker initialization handler to avoid this bug happens. Because the nginx worker initialization handler will always be the first registered handler which need not handle any request body. e.g.

For clojure

http {
    handler_type 'clojure'; 
    handler_code '(fn[req] nil)';  
}

For java

http {
    handler_type 'java'; 
    handler_name 'my.MyNothingToDoHandler';  
}

public class MyNothingToDoHandler implements NginxJavaRingHandler {
   public Object[] invoke(Map<String, Object> request){
       return null;
   }
}

Usage Issue

I have tried to compile nginx-clojure via Ubuntu 12.04's build tool, but everytime I try to hit the /clojure location, I have a Method not found exception.

Oddly, I have a 16KB .jar file, compared to the binary one which is 42KB. I have placed the jars into /etc/nginx/jars.

Exception in thread "main" java.lang.NoSuchMethodError: Method nginx.clojure.NginxClojureRT.ngx_create_file_buf(JJJI)J not found
2014/01/19 16:45:30 [error] 31648#0: can not initialize jvm memory util

Any Ideas?

Classpath expansion

Is there a way to have nginx or nginx-clojure do a (single-depth is fine) pathname search and expand that to something that could be put in the -Djava.class.path= portion of jvm_options? I've got a bunch of jars in three directories (one directory for the nginx-clojure core classpath (includes nginx-clojure-0.2.2.jar, clojure-1.6.0.jar, and others), and two for .jar files for two individual users (the only two users who currently use the nginx daemon). However, I have to manually specify each .jar since java.class.path doesn't support wildcard expansion for JARs. It would be nice to have something search the three directories (again, it can be single-depth, I don't need to recurse subdirectories) and generate a list of JARs separated by ':' (or ';', in the case of Windows) that could be assigned to a variable to be passed in to jvm_options or included in jvm_options by way of a subexpression or replacement symbol. Let me know if that would be possible.

Provide wrapper of asynchronous socket to make the usage of them easier

So far there hasn't any wrapper of asynchronous socket. But asynchronous socket is a little too low level to use them easily. So we need a wrapper of asynchronous socket to make the usage of them easier. This wrapper should also consider the consistent problems with future WebSocket/SSE(Server Sent Event) API provided by Nginx-Clojure.

Configurable Lazy or Eager Initialization of Handlers

All handlers are lazy Initialized. This mean it won't be Initialized until a related request incoming.
It is not so friendly to check the handlers eagerly.

To resovle this problem we should add a new nginx directive handlers_lazy_init which can be used at the nginx.conf blocks of http / server / location and its default value is off.

After invoking on coroutine based socket nginx worker will exit and be recreated when network is disabled

This issue can be reproduced after these steps

1. enable coroutine based socket
2. disable network
3. socket.connect

Then nginx worker will exit and be recreated.

Auto generated waving class configurations about Proxy InvocationHandler instance

e.g.

package nginx.clojure.demo;

    public  interface MyService {
        public int compute(int a, int b) throws SuspendExecution;
    }

.....

/*in some method*/
MyService remoteService =(MyService)Proxy.newProxyInstance
(loader, new Class<?>[]{MyService.class}, new InvocationHandler() {

            @Override
            public Object invoke(Object proxy, Method method, Object[] args)
                    throws Throwable, SuspendExecution {
                //invoke remote http service & return result;
            }
        });

remoteService.compute(1, 2);

When we turn on running tool mode to generate waving class configurations, after running above code and dumping the waving class configurations, the generated waving class configurations should contains MyService/remoteCompute waving infomation which is like

lazyclass:nginx/clojure/demo/MyService
  compute(II)I:normal

The more general case is that when we use Hessian Service Client HessianProxyFactory will create proxy instance to access remote service by URLConnection. If nginx-clojure can auto generate waving class configurations about Proxy InvocationHandler instance, we can easily make Hessian Service Client run on the coroutine based socket on nginx without blocking.

With Compojure 1.1.5 + Apache Solrj 4.3.0 + httpclient 4.3.2 NPE happens first time then everything is OK

After enable coroutine based socekt with Solrj 4.3.0 + httpclient 4.3.2, the first invoking meets NPE, then all of the next invokings are OK.

The NPE exception stacktraces are here

org.apache.solr.client.solrj.SolrServerException: Error executing query
    at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:98)
    at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
    at test.core$test_solr_handler.invoke(core.clj:56)
    at clojure.lang.AFn.applyToHelper(AFn.java:178)
    at clojure.lang.AFn.applyTo(AFn.java:151)
    at clojure.core$apply.invoke(core.clj:623)
    at clojure.core$partial$fn__4196.doInvoke(core.clj:2402)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at compojure.response$fn__151.invoke(response.clj:27)
    at compojure.response$fn__128$G__123__135.invoke(response.clj:10)
    at compojure.core$make_route$fn__346.invoke(core.clj:93)
    at compojure.core$if_route$fn__334.invoke(core.clj:39)
    at compojure.core$if_method$fn__327.invoke(core.clj:24)
    at compojure.core$routing$fn__352.invoke(core.clj:106)
    at clojure.core$some.invoke(core.clj:2443)
    at compojure.core$routing.doInvoke(core.clj:106)
    at clojure.lang.RestFn.applyTo(RestFn.java:139)
    at clojure.core$apply.invoke(core.clj:619)
    at compojure.core$routes$fn__356.invoke(core.clj:111)
    at ring.middleware.params$wrap_params$fn__738.invoke(params.clj:58)
    at compojure.core$routing$fn__352.invoke(core.clj:106)
    at clojure.core$some.invoke(core.clj:2443)
    at compojure.core$routing.doInvoke(core.clj:106)
    at clojure.lang.RestFn.invoke(RestFn.java:423)
    at test.core$fn__6155.invoke(core.clj:86)
    at compojure.core$wrap_context$fn__370.invoke(core.clj:164)
    at compojure.core$if_route$fn__334.invoke(core.clj:39)
    at compojure.core$routing$fn__352.invoke(core.clj:106)
    at clojure.core$some.invoke(core.clj:2443)
    at compojure.core$routing.doInvoke(core.clj:106)
    at clojure.lang.RestFn.applyTo(RestFn.java:139)
    at clojure.core$apply.invoke(core.clj:619)
    at compojure.core$routes$fn__356.invoke(core.clj:111)
    at nginx.clojure.NginxClojureRT$CoroutineRunner.run(NginxClojureRT.java:176)
    at nginx.clojure.Coroutine.resume(Coroutine.java:198)
    at nginx.clojure.net.NginxClojureSocketImpl.onConnect(NginxClojureSocketImpl.java:348)
    at nginx.clojure.net.NginxClojureAsynSocket.onConnect(NginxClojureAsynSocket.java:266)
Caused by: java.lang.NullPointerException
    at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:352)
    at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
    at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:90)

Configuration check JVM

It would be great to initialize the plugin only if - for example - "jvm_path" is configured. I am creating a custom Nginx build for different server configurations and not on all Java is available (or desired/needed).

With the current setup nginx-clojure immediately checks for the jvm_path and fails if it's not provided. This forces me to create different Nginx distributions for Java and non-Java enabled servers.

Alternatively there could be also a new parameter:

clojure on;

that would indicate if the clojure module should be initialized or not (similar to "gzip on;").

Support Groovy - another dynamic jvm language

Just like Clojure We can use either an external Groovy Handler or inline Groovy code in nginx.conf.

1. External Groovy Handler

   location /groovy2 {
      handler_type 'groovy';
      handler_name 'nginx.clojure.groovy.HelloGroovy2';
   }

in HelloGroovy2.groovy

package nginx.clojure.groovy;

import nginx.clojure.java.NginxJavaRingHandler;
import java.util.Map;

public class HelloGroovy2 implements NginxJavaRingHandler {
    public Object[] invoke(Map<String, Object> request){
       return 
         [ 200,  //http status
           ["Content-Type":"text/html"],  //headers map
           "Hello, Groovy2 & Nginx!" //response body can be string, File or Array/Collection of string or File
        ];
    }
 }

2. Inline Groovy code

   location /groovy {
      handler_type 'groovy';
      handler_code '
          import nginx.clojure.java.NginxJavaRingHandler;
          import java.util.Map;
          public class HelloGroovy implements NginxJavaRingHandler {
             public Object[] invoke(Map<String, Object> request){
                 return [200, ["Content-Type":"text/html"], "Hello, Groovy & Nginx!"];
             }
          }
        ';
     } 

proxy pass location

Hi ,

I have few questions ,

can the inline ring handler accomodate the proxy_pass location's and changes in the headers propagated to the proxy server , and proxy_buffers as well

What are the phases of nginx where handlers operate at .

Support user defined http request method

Some web services need user defined http request method to define special operations beyond standard http request methods.

e.g. We use MYUPLOAD to upload a file and overwrite the one if it exists. The curl command maybe is

curl   -v  -X MYUPLOAD  --upload-file post-test-data \
"http://localhost:8080/myservice" 

In the nginx.conf, we can use always_read_body on; to force nginx to read http body.

location /myservice {
         clojure;
         always_read_body on;
         clojure_code '....';
}

Must specifiy jvm_options "-Djava.awt.headless=true"; in nginx.conf when importing Swing

I imagine this is not something you'll want to fix (nor whether it occurs outside of OSX), but it was a nasty that took me a bit to track down since without this option it looks like nginx hangs.

Tip 1:
Also found that by adding the location of your clojure source files to the classpath; e.g. jvm_options "-Djava.class.path=/Users/me/src" you can then just issue "nginx -s reload" and changes to the sources get picked up!

Tip 2:
You can remove clojure-1.5.1.jar from class path and point at your "lein uberjar" to pick up a different version of clojure.

Tip 3:
To use Java 7 on OSX (in nginx.conf)
jvm_path "/Library/Java/JavaVirtualMachines/jdk1.7.0_55.jdk/Contents/Home/jre/lib/server/libjvm.dylib";

reading from coroutine based socket inputstream returns 0 when eof, should return -1

Java InputStream API document said

int read(byte b[], int off, int len) throws IOException

Reads up to len bytes of data from the input stream into an array of bytes. An attempt 
is made to read as many as len bytes, but a smaller number may be read. The 
number of bytes actually read is returned as an integer.

This method blocks until input data is available, end of file is detected, or an exception 
is thrown.

If len is zero, then no bytes are read and 0 is returned; otherwise, there is an attempt 
to read at least one byte. If no byte is available because the stream is at end of file, 
the value -1 is returned; otherwise, at least one byte is read and stored into b.

So only if len == 0 we can returns 0, otherwise we should return -1 when real socket returns 0.

nginx shared dict

Does the nginx-clojure module provide methods to read/write shared nginx dictionaries?

agent library failed to init: instrument

Thanks for working on this project, we've been enjoying it. I'm trying to create a waving file for my project but I keep getting the error in the subject when I start nginx. Here's a bit of my config:

http {
...
#turn on run tool mode, t means Tool
jvm_options "-javaagent:/opt/fast_and_furious/nginx-clojure-0.2.2.jar=tmb";
jvm_options "-Xbootclasspath/a:/home/vagrant/src/nginx-clojure/target/nginx-clojure-0.2.2.jar:/home/vagrant/.m2/repository/org/clojure/clojure/1.5.1/clojure-1.5.1.jar";
}

$ /usr/local/nginx/sbin/nginx
agent library failed to init: instrument

non-blocking socket compatible with largely existing java libraries such as jdbc drivers

This socket implementation have all features including

(1) non-blocking
(2) coroutine based so one main thread can handle thousands of connections
(3) be friendly to largely existing java libraries such as jdbc drivers so we can dev & test on lein ring and deploy with nginx-clojure without change any code

Coroutine based socket supports JDK8

Coroutine based socket has dependency on class waving by re-rooted asm 4.2 which doesn't support JDK8. So we need to upgrade it to 5.0.3..

Access request BODY in rewrite handler

Can you please make it possible to access the request body in a JAVA rewrite handler.
This way I can route traffic based on a specific value in the body.

Using the asynchronous socket mode

Hi, I am very interested in using the async sockets you provide to the client libs. If I use them with core.async, will core.async still initialize its own thread pool and run its go blocks there? If not, could you make an example of the async sockets used in Clojure?

Request body buffering

It seems any request body larger than the default client_body_buffer_size is buffered to a flat file (as expected), however, the buffer is cleaned before the request is handed to clojure. What is the recommended way to deal with this? I would rather not set client_body_buffer_size to match client_max_body_size as this will harm server performance under load.

2014/05/13 02:37:47 [warn] 32248#0: *26847 a client request body is buffered to a temporary file /var/lib/nginx/body/0000001310, client: 127.0.0.1, server: , request: "PUT /test HTTP/1.1", host: "example.com"
server unhandled exception!
java.lang.RuntimeException: can not find tmp file
        at nginx.clojure.RequestBodyFetcher.fetch(RequestBodyFetcher.java:32)
        at nginx.clojure.LazyRequestMap.element(LazyRequestMap.java:217)
        at nginx.clojure.LazyRequestMap.valAt(LazyRequestMap.java:232)
        at clojure.lang.KeywordLookupSite$1.get(KeywordLookupSite.java:45)
        at ring.middleware.params$assoc_form_params.invoke(params.clj:28)
        at ring.middleware.params$wrap_params$fn__450.invoke(params.clj:51)
        at ring.middleware.multipart_params$wrap_multipart_params$fn__545.invoke(multipart_params.clj:103)
        at ring.middleware.flash$wrap_flash$fn__1194.invoke(flash.clj:14)
        at ring.middleware.session$wrap_session$fn__1183.invoke(session.clj:43)
        at ring.middleware.cookies$wrap_cookies$fn__1114.invoke(cookies.clj:160)
        at test.handler$raw_handler.invoke(handler.clj:81)
        at nginx.clojure.NginxClojureRT.handleRequest(NginxClojureRT.java:507)
        at nginx.clojure.NginxClojureRT.eval(NginxClojureRT.java:486)
Caused by: java.io.FileNotFoundException: /var/lib/nginx/body/0000001310 (No such file or directory)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(FileInputStream.java:146)
        at java.io.FileInputStream.<init>(FileInputStream.java:101)
        at nginx.clojure.RequestBodyFetcher.fetch(RequestBodyFetcher.java:30)
        ... 12 more

Supports Server Sent Events (SSE)

More details can be found from http://www.w3.org/TR/eventsource/ .
We'll provide related APIs to support server side SSE.

More friendly to java users who maybe know nothing about clojure

Not all Java users are familiar with Clojure. With this feathure java users can declare java handler simply, e.g.

    handler_type 'java';
    handler_name 'my.HelloHandler';

And in the source of HelloHandler no clojure class need be imported

import java.util.Map;
import static nginx.clojure.MiniConstants.*;
import nginx.clojure.java.ArrayMap;
import nginx.clojure.java.NginxJavaRingHandler;

public static class HelloHandler implements NginxJavaRingHandler {

    @Override
public Object[] invoke(Map<String, Object> request) {
    return new Object[] { 
          NGX_HTTP_OK,  //http status 200
          ArrayMap.create(CONTENT_TYPE, "text/plain"),  //headers map
          "Hello, Java & Nginx!"  //response body can be string, File or Array/Collection of string or File
        };
 }
}

The above code is more friendly to java users who maybe know nothing about clojure.

PATCH loses the data payload

An incoming PATCH request has its data payload truncated, and Content-Length set to 0.

Supports broadcast events to all Nginx workers

Suppose our Nginx instance has 4 workers (worker process not jvm_workers which is just thread number of thread pool in jvm). Now we want to provide sub/pub service. e.g.

1. Client A connected to nginx worker A and subscribed to uri /mychannel/sub
2. Client B connected to nginx worker B and subscribed to uri /mychannel/sub
3. Client C connected to nginx worker C and publish a message to uri /mychannel/pub

So the service at endpoint of /mychannel/pub must broadcast pub event to Client A and Client B.
Although for large-scale application we can use sub/pub service from Redis on nginx-clojure , for small-scale or medium-scale application this feature will make the dev life easier.

This feature will support two kinds of events to broadcast, simple events and complex events.

1. A simple event only has a event id which is a long integer and must be less than 0x0100000000000000L, it hasn't any body or its body is stored in some external stores, e.g. SharedHashMap, Memcached, Redis etc.
2. A complex event has a message with a length limitation PIPE_BUF - 8, generally on Linux/Windows is 4088, on MacosX is 504.

rewrite handler does not handle write event correctly with thread pool mode or coroutine mode

This issue can be reproduced after these steps

1. enable thread pool mode or coroutine mode
2. set keepalive_timeout to be 65
3. invoke remote service in rewrite handler
4. set some nginx variable
5. proxy to some upstream according to some nginx variable

If an additional write event happens before rewrite handler's completion it will cause nginx error or worker process to exit and be recreated.

More friendly and high level long polling API

Although we can use low level API such as ASYNC_TAG, completeAsyncResponse to do long polling without any problem. But we need more friendly high level API which should be consistent with future WebSocket/SSE(Server Sent Event) API provided by Nginx-Clojure.

Slow Memory Leak for Coroutine based Socket

The C Release handler doesn't release the native global reference of socket object unless there 's an exception threw by Java release handler.

static void  jni_ngx_http_clojure_socket_release_handler(ngx_http_clojure_socket_upstream_t *u, ngx_int_t sc) {
    (*jvm_env)->CallVoidMethod(jvm_env, (jobject)u->context, nc_socket_handler_release_mid, (jlong)(uintptr_t)u, (jlong)sc);
    exception_handle(0 == 0, jvm_env, (*jvm_env)->DeleteGlobalRef(jvm_env, (jobject)u->context));
}

The last line will cause slow memory leak for coroutine based socket.

It should be

static void  jni_ngx_http_clojure_socket_release_handler(ngx_http_clojure_socket_upstream_t *u, ngx_int_t sc) {
    (*jvm_env)->CallVoidMethod(jvm_env, (jobject)u->context, nc_socket_handler_release_mid, (jlong)(uintptr_t)u, (jlong)sc);
    exception_handle(0 == 0, jvm_env, (*jvm_env)->DeleteGlobalRef(jvm_env, (jobject)u->context);return);
    (*jvm_env)->DeleteGlobalRef(jvm_env, (jobject)u->context);
}

Handle multiple sockets parallel in sub coroutines, e.g. we can invoke two remote services at the same time

Sometimes we need invoke serveral remote services before completing the ring response. For better performance we need a way to handle multiple sockets parallel in sub coroutines.

e.g. fetch two page parallel by clj-http

   (let [[r1, r2] 
                (co-pvalues (client/get "http://service1-url") 
                            (client/get "http://service2-url"))]
    ;println bodies of two remote response
    (println (str (:body r1) "====\n" (:body r2) ))

Here co-pvalues is also non-blocking and coroutine based. In fact it will create two sub coroutines to handle two sockets.

BTW. It only works when coroutine based socket mode enabled.