openresty / drizzle-nginx-module

an nginx upstream module that talks to mysql and drizzle by libdrizzle

Name

OpenResty - Turning Nginx into a Full-Fledged Scriptable Web Platform

Table of Contents

• Name
• Description
  • For Users
  • For Bundle Maintainers
• Additional Features
  • resolv.conf parsing
• Mailing List
• Report Bugs
• Copyright & License

Description

OpenResty is a full-fledged web application server by bundling the standard nginx core, lots of 3rd-party nginx modules, as well as most of their external dependencies.

This bundle is maintained by Yichun Zhang (agentzh).

Because most of the nginx modules are developed by the bundle maintainers, it can ensure that all these modules are played well together.

The bundled software components are copyrighted by the respective copyright holders.

The homepage for this project is on openresty.org.

For Users

Visit the download page on the openresty.org web site to download the latest bundle tarball, and follow the installation instructions in the installation page.

For Bundle Maintainers

The bundle's source is at the following git repository:

https://github.com/openresty/openresty

To reproduce the bundle tarball, just do

make

at the top of the bundle source tree.

Please note that you may need to install some extra dependencies, like perl, dos2unix, and mercurial. On Fedora 22, for example, installing the dependencies is as simple as running the following commands:

sudo dnf install perl dos2unix mercurial

Back to TOC

Additional Features

In additional to the standard nginx core features, this bundle also supports the following:

Back to TOC

resolv.conf parsing

syntax: resolver address ... [valid=time] [ipv6=on|off] [local=on|off|path]

default: -

context: http, stream, server, location

Similar to the resolver directive in standard nginx core with additional support for parsing additional resolvers from the resolv.conf file format.

When local=on, the standard path of /etc/resolv.conf will be used. You may also specify arbitrary path to be used for parsing, for example: local=/tmp/test.conf.

When local=off, parsing will be disabled (this is the default).

This feature is not available on Windows platforms.

Back to TOC

Mailing List

You're very welcome to join the English OpenResty mailing list hosted on Google Groups:

https://groups.google.com/group/openresty-en

The Chinese mailing list is here:

https://groups.google.com/group/openresty

Back to TOC

Report Bugs

You're very welcome to report issues on GitHub:

https://github.com/openresty/openresty/issues

Back to TOC

Copyright & License

The bundle itself is licensed under the 2-clause BSD license.

Copyright (c) 2011-2019, Yichun "agentzh" Zhang (章亦春) [email protected], OpenResty Inc.

This module is licensed under the terms of the BSD license.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

• Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
• Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Back to TOC

drizzle-nginx-module's Issues

failed to connect: 10: drizzle_state_handshake_server_read:bad packet size:>=46:23 while connecting to drizzle upstream

Any idea what would cause this error? Serving 1000's of req/sec without issue and then this popped up. Too many connections to the backend? (This is definitely not mysql 5.5, it's 5.1.)

The drizzle_status output looks like:

active connections: 0
connection pool capacity: 200
overflow: ignore
cached connection queue: 0
free'd connection queue: 200
cached connection successfully used count:
free'd connection successfully used count: 2 4 2 3 2 4 4 4 2 4 2 4 3 2 4 2 4 3 3 4 3 4 4 4 4 4 4 4 4 3 4 4 4 4 3 4 4 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 1 2 2 2 2 2 2 2 2 2 2 3 4 2 6 4 4 3 2 4 2 4 4 2 2 4 2 3 2 2 3 3 4 4 2 2 3 2 2 3 3 4 4 4 4 4 2 3 2 4 4 3 2 4 3 4 2 2 2 2 4 3 2 2 3 4 3 2 2 4 3 2 2 2 2 4 2 2 2 2 2 3 3 2 4 2 4 3 2 4 4 2 2 2 4 4 3 4 2 3 2 3 7 6 2 2 3 2 3 2 2 2 3 2 5 7 4 2 3 3 4 2 2 4 4 2 2 4 2 2 4 2 4 3 2 2 2 4 2 2
servers: 1
peers: 1

nginx的drizzle中，获取(查询)多个列值时报错double free or corruption (!prev): 0x000000000543a9e0

nginx的drizzle中，获取(查询)多个列值时报错如下
*** glibc detected *** /root/work/nginx: double free or corruption (!prev): 0x000000000543a9e0 ***
ngx_http_upstream_drizzle_send_query--》ngx_http_upstream_drizzle_recv_cols

在ngx_http_upstream_drizzle_recv_cols函数中调用drizzle_column_free报错！两次释放同一地址。

Latest Nginx 1.3.15 (From FreeBSD Ports) Breaks build

When trying to build via the Nginx-Devel port on FreeBSD 9.1, (previously 1.3.14 successfully), the following error is given.

/var/ports/usr/ports/www/nginx-devel/work/chaoslawful-drizzle-nginx-module-59e12ad/src/ngx_http_drizzle_util.c: In function 'ngx_http_upstream_dbd_init_request':
/var/ports/usr/ports/www/nginx-devel/work/chaoslawful-drizzle-nginx-module-59e12ad/src/ngx_http_drizzle_util.c:437: warning: assignment from incompatible pointer type
/var/ports/usr/ports/www/nginx-devel/work/chaoslawful-drizzle-nginx-module-59e12ad/src/ngx_http_drizzle_util.c: In function 'ngx_http_upstream_dbd_connect':
/var/ports/usr/ports/www/nginx-devel/work/chaoslawful-drizzle-nginx-module-59e12ad/src/ngx_http_drizzle_util.c:584: error: 'ngx_connection_t' has no member named 'single_connection'
*** [objs/addon/src/ngx_http_drizzle_util.o] Error code 1

Will drizzle_nginx_module support mysql ssl connection?

My project need ssl connection with mysql. Will drizzle_nginx_module support ssl connection to mysql?

Or how can I deal with the code to make it done?

How to configure ngx_drizzle for SQLite3

In README , it said :
ngx_drizzle - an upstream module that talks to
mysql, drizzle, and sqlite3 by libdrizzle

So , can you provides an example for sqlite3 ?

drizzle fails to compile on latest nginx

/home/eklinger/Downloads/drizzle-nginx-module//src/ngx_http_drizzle_output.c: In function ‘ngx_http_drizzle_output_bufs’:
/home/eklinger/Downloads/drizzle-nginx-module//src/ngx_http_drizzle_output.c:290:17: error: passing argument 1 of ‘ngx_chain_update_chains’ from incompatible pointer type [-Werror]
src/core/ngx_buf.h:157:6: note: expected ‘struct ngx_pool_t ’ but argument is of type ‘struct ngx_chain_t *’
/home/eklinger/Downloads/drizzle-nginx-module//src/ngx_http_drizzle_output.c:290:17: error: too few arguments to function ‘ngx_chain_update_chains’

Two SQLs to execute in one time and then throw an error.

does it support the multi sql block?

crash

ngx_http_drizzle_create_loc_conf(ngx_conf_t *cf){
conf->complex_target = NGX_CONF_UNSET_PTR;//0xffffffffffffffff
}

ngx_int_t
ngx_http_drizzle_handler(ngx_http_request_t r){
if (dlcf->complex_target) {//0xffffffffffffffff
/ variables used in the drizzle_pass directive */
if (ngx_http_complex_value(r, dlcf->complex_target, &target)
!= NGX_OK) //cash here..
{
dd("failed to compile");
return NGX_ERROR;
}
}

Troubleshoot drizzle connection

I am using openresty with the drizzle module. I have everything running fine on my mac. I installed the same thing on a Ubuntu EC2 instance, and everything is working except the database connection to drizzle. Any subrequest, or direct request that goes through the /mysql location returns a 200 with 0 data. If I go to /mysql_pool_status, the same thing. 200 with 0 data. I get no error message in error.log.

The /mysql location is:location

/mysql {
rds_json on;
drizzle_query $arg_sql;
default_type application/json;
drizzle_connect_timeout 500ms;
drizzle_send_query_timeout 2s;
drizzle_pass backend;
}

upstream backend {
drizzle_server 127.0.0.1:3306 dbname=identity user=root protocol=mysql;
}

I access the /mysql location using ngx.location.capture. In this case, it never returns, but looks like it exits. Again with a 200 and 0 data.

I can connect to the database through port 3306 in other ways.

My problem is that I can't trouble shoot this because there are no errors in error.log. How should I proceed.

Incidentally, it's running on Hardy and openresty 1.0.4, drizzle 7-2011.08.24, and mysql 5.0.

Nginx 1.3.15 build fails with drizzle module

When trying to build via the Nginx-Devel port on FreeBSD 9.1, (previously 1.3.14 successfully), the following error is given.

/var/ports/usr/ports/www/nginx-devel/work/chaoslawful-drizzle-nginx-module-59e12ad/src/ngx_http_drizzle_util.c: In function 'ngx_http_upstream_dbd_init_request':
/var/ports/usr/ports/www/nginx-devel/work/chaoslawful-drizzle-nginx-module-59e12ad/src/ngx_http_drizzle_util.c:437: warning: assignment from incompatible pointer type
/var/ports/usr/ports/www/nginx-devel/work/chaoslawful-drizzle-nginx-module-59e12ad/src/ngx_http_drizzle_util.c: In function 'ngx_http_upstream_dbd_connect':
/var/ports/usr/ports/www/nginx-devel/work/chaoslawful-drizzle-nginx-module-59e12ad/src/ngx_http_drizzle_util.c:584: error: 'ngx_connection_t' has no member named 'single_connection'
*** [objs/addon/src/ngx_http_drizzle_util.o] Error code 1

More paths for path libdrizzle path disocvery

Ubuntu 12.04 (LTS) installs libdrizzle into /usr/lib/ and /usr/include/libdrizzle-1.0. By default the configure script does not search this path.

Could these be added?

If anyone stumbles accross this ticket with this problem the interim solution is to call configure prefixed with:

env LIBDRIZZLE_LIB=/usr/lib/ LIBDRIZZLE_INC=/usr/include/libdrizzle-1.0 [...]

Compilation issue with bool on OSX

I am compiling nginx under OSX and this is the error message I am getting.

gcc -c -pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Wunused-function -Wunused-variable -Wunused-value -Werror -g -I src/core -I src/event -I src/event/modules -I src/os/unix -I /Library/PostgreSQL/8.4/include/ -I objs -I src/http -I src/http/modules -I src/mail
-o objs/addon/src/ngx_http_drizzle_module.o
../chaoslawful-drizzle-nginx-module-f0ba8f5/src/ngx_http_drizzle_module.c
In file included from /usr/local/include/libdrizzle/drizzle.h:42,
from /usr/local/include/libdrizzle/drizzle_client.h:19,
from ../chaoslawful-drizzle-nginx-module-f0ba8f5/src/ngx_http_drizzle_module.h:8,
from ../chaoslawful-drizzle-nginx-module-f0ba8f5/src/ngx_http_drizzle_module.c:7:
/usr/local/include/libdrizzle/result.h:69: error: syntax error before ‘drizzle_result_eof’
make[1]: *** [objs/addon/src/ngx_http_drizzle_module.o] Error 1
make: *** [build] Error 2

Line 69 is DRIZZLE_API
bool drizzle_result_eof(drizzle_result_st *result);

OS = uname -a
Darwin sahuguet-macbookpro.local 9.8.0 Darwin Kernel Version 9.8.0: Wed Jul 15 16:55:01 PDT 2009; root:xnu-1228.15.4~1/RELEASE_I386 i386

regards,

Arnaud

Example of Select + Insert

Can you provide an example of a selecting a record, and then issuing an insert if the record does not exist? Is it possible to accomplish this with a single location and multiple queries?

resolver support for Amazon RDS

With Amazon RDS, the ip address can change after certain modifications (upgrade to more powerful instance, add more disk space, etc). It seems like drizzle-nginx-module resolves the hostname on startup only. Is it possible to get it to use http://nginx.org/en/docs/http/ngx_http_core_module.html#resolver?

Otherwise, I think I can live with writing a bash script that does periodic dig and reloads nginx when the address changes, just to continue using openresty :)

the column names are wrong when doing union select for two tables

when I have two tables one for table_a

id	name
1	Jack

the other table_b

id	email
1	[email protected]

when I do union select:

select * from table_a where id=1   union select * from table_b where id=1

I get the response body

[{"id":"1","name":"Jack"},{"id":"1","name":"[email protected]"}]

the column name email was replaced by name in the 2nd element of Json output.

drizzle_status doesn't appear to work

When configuring as described in the documentation (and running nginx -c nginx.conf -t):

[emerg]: unknown directive "drizzle_status" in nginx.conf:32

How to change the return value

hello ：

this is my conf

location /upload{
            root  /usr/local/software/appstorm;
           #rewrite ^upload/(.*)$ /usr/local/nginx/html/$1 last;
            #root /usr/local/nginx/html;
            index index.html index.htm;
     }


    location / {
        set $clientId $arg_clientId;
        set $id $arg_id;
        #set_quote_sql_str  $quoted_clientId $clientId;
        #set_quote_sql_str $quoted_id $id;
        #drizzle_query "select * from t_upgrade_user";
        drizzle_query "INSERT INTO t_upgrade_user(p_client_id,p_lib_id) values('$clientId','$id')";
        drizzle_pass mysqlbackend;
        drizzle_module_header off;
        #rds_json on;
        #root   /usr/local/software/appstorm;
        #root /usr/local/nginx/html;
        #index  index.html index.htm;
        rewrite ^/(.*)$ /upload/$1 last;
    }

Can be successfully inserted into the MySQL database, but as you can see I do not want to insert the successful state returned to client. I want to get the download path for root.

What can I do?

编译 libdrizzle-1.0出错，请帮忙看看

章工：

我参考 http://wiki.nginx.org/HttpDrizzleModule#Installation 上面的内容 安装HttpDrizzleModule 会出现错误，具体信息如下：

./configure --without-server 后的信息

Configuration summary for drizzle7 version 7 drizzle

• Installation prefix: /usr/local/drizzle
• System type: unknown-linux-gnu
• pandora-build version: 0.175
• Host CPU: x86_64
• C Compiler: gcc44 (GCC) 4.4.6 20110731 (Red Hat 4.4.6-3)
• C++ Compiler: g++44 (GCC) 4.4.6 20110731 (Red Hat 4.4.6-3)
• Assertions enabled: yes
• Debug enabled: no
• Profiling enabled: no
• Coverage enabled: no
• Warnings as failure: no

make libdrzzle-1.0 后的信息

cd . && python config/pandora-plugin write
CC libdrizzle-1.0/libdrizzle/libdrizzle_1_0_libdrizzle_libdrizzle_la-column.lo
CC libdrizzle-1.0/libdrizzle/libdrizzle_1_0_libdrizzle_libdrizzle_la-command.lo
CC libdrizzle-1.0/libdrizzle/libdrizzle_1_0_libdrizzle_libdrizzle_la-conn.lo
CC libdrizzle-1.0/libdrizzle/libdrizzle_1_0_libdrizzle_libdrizzle_la-conn_uds.lo
CC libdrizzle-1.0/libdrizzle/libdrizzle_1_0_libdrizzle_libdrizzle_la-drizzle.lo
CC libdrizzle-1.0/libdrizzle/libdrizzle_1_0_libdrizzle_libdrizzle_la-field.lo
CC libdrizzle-1.0/libdrizzle/libdrizzle_1_0_libdrizzle_libdrizzle_la-handshake.lo
CC libdrizzle-1.0/libdrizzle/libdrizzle_1_0_libdrizzle_libdrizzle_la-pack.lo
CC libdrizzle-1.0/libdrizzle/libdrizzle_1_0_libdrizzle_libdrizzle_la-query.lo
CC libdrizzle-1.0/libdrizzle/libdrizzle_1_0_libdrizzle_libdrizzle_la-result.lo
CC libdrizzle-1.0/libdrizzle/libdrizzle_1_0_libdrizzle_libdrizzle_la-row.lo
CC libdrizzle-1.0/libdrizzle/libdrizzle_1_0_libdrizzle_libdrizzle_la-sha1.lo
CC libdrizzle-1.0/libdrizzle/libdrizzle_1_0_libdrizzle_libdrizzle_la-state.lo
CCLD libdrizzle-1.0/libdrizzle/libdrizzle.la

make install libdrizzle-1.0 的报错信息

cd . && python config/pandora-plugin write
Making drizzled/configmake.h
no --proto_path=./drizzled/message:./drizzled/message
--cpp_out=./drizzled/message drizzled/message/alter_table.proto
make: no：命令未找到
make: *** [drizzled/message/alter_table.pb.h] 错误 127

另外，我之前在一台服务器上编译安装成功过，不过现在重新安装也出现这个错误。
系统是centos 5.5 x86_64 已经更新到了 最新版本。

前些日子给你发过邮件，但是由于无法登录openresty的google论坛所以一直没有致谢。
在此一并感谢。

Fails to Build on Nginx 3.1.16

The following error is generated when trying to build /usr/ports/www/nginx-devel (version 3.1.16).

cc -c -O2 -pipe -fno-strict-aliasing -I /usr/local/include -I src/core -I src/event -I src/event/modules -I src/os/uni x -I objs -I src/http -I src/http/modules -I src/mail -o objs/addon/src/ngx_http_drizzle_util.o /var/ports/usr/ports /www/nginx-devel/work/chaoslawful-drizzle-nginx-module-59e12ad/src/ngx_http_drizzle_util.c
/var/ports/usr/ports/www/nginx-devel/work/chaoslawful-drizzle-nginx-module-59e12ad/src/ngx_http_drizzle_util.c: In functio n 'ngx_http_upstream_dbd_init_request':
/var/ports/usr/ports/www/nginx-devel/work/chaoslawful-drizzle-nginx-module-59e12ad/src/ngx_http_drizzle_util.c:437: warnin g: assignment from incompatible pointer type
/var/ports/usr/ports/www/nginx-devel/work/chaoslawful-drizzle-nginx-module-59e12ad/src/ngx_http_drizzle_util.c: In functio n 'ngx_http_upstream_dbd_connect':
/var/ports/usr/ports/www/nginx-devel/work/chaoslawful-drizzle-nginx-module-59e12ad/src/ngx_http_drizzle_util.c:584: error: 'ngx_connection_t' has no member named 'single_connection'
*** [objs/addon/src/ngx_http_drizzle_util.o] Error code 1

Stop in /var/ports/usr/ports/www/nginx-devel/work/nginx-1.3.16.
*** [build] Error code 1

Stop in /var/ports/usr/ports/www/nginx-devel/work/nginx-1.3.16.
*** [do-build] Error code 1

Stop in /usr/ports/www/nginx-devel.
*** [build] Error code 1

Stop in /usr/ports/www/nginx-devel.

This is on a fresh jail that has not had nginx installed on it before, so no previous dist files or dependencies (/usr/ports/databases/libdrizzle was installed prior).

I'm somewhat concerned that I'll need to stay on version 1.2 for a while, as Nginx 1.4 was just released not that long ago for both stable/dev branch, and have not seen a fix for this yet.

why not support different databases and tables linked sql

for example : select u.name from db1.user u, db2.user u1 where u.id = u1.id

keepalive_get will reuse connections with state_db_connect

in v1.1.0

conf is
upstream dns_backend {
drizzle_server 10.232.31.89:3306;
drizzle_keepalive max=70 mode=single overflow=ignore;
}

we find that the connection can be only used once.
and the error log says "BROKEN PIPE"

we found that the ngx_http_drizzle_keepalive_get_XXX() returns the dp with reset dp->state, so it is state_db_connect

how can i execute mysql procedure in drizzle_query

like:
set $my_sql 'call p_query()';
drizzle_query $my_sql;

HTTP/2 supporting

Since lua_nginx_module been support HTTP/2: openresty/lua-nginx-module#566
Is there any chance we could also get drizzle-nginx-module to work with HTTP/2 ?

Broken link to Drizzle-nginx-module info page

Interestingly, the link on the front repo page seems to be broken, going to a generic new Wordpress page:

Guessing it's supposed to be pointing at project docs or something instead?

For reference, this is the broken link:

ngx.location.capture 内部请求乱码

请教您一个问题，我目前遇到的问题我没有想到原因，希望您有时间可以帮我看一下。
目前我配置文 drizzle ，在 nginx 配置好了 location 直接访问是可以的，但是通过 ngx.location.capture 访问 有乱码。搞不清楚原因。

           nginx 配置文件：
                   upstream mysql {
                              drizzle_server 127.0.0.1:3306 dbname=cinema user=root password=123456 protocol=mysql charset=utf8;
                              drizzle_keepalive max=200 overflow=ignore mode=single;
                     }

 location /abc {
    set_quote_sql_str $id $arg_id;
    drizzle_query 'select c_ip as CIP,c_name as CName,c_num as CNum from c_info where c_status=0 and c_num in (14033071,44012501)';
    drizzle_pass mysql;
    rds_json on;
}

    location /qwe {
         default_type 'text/plain';
         content_by_lua_file '/data0/app/nginx/conf/lua/get_sql.lua';
    }

      cat get_sql.lua >>>

        local res = ngx.location.capture("/abc")
       if res.status == 200 then
             ngx.say(res.body)
       end

        nginx  版本   Tengine/2.1.2             

curl "http://127.0.0.1/abc"
[{"CIP":"10.255.126.34","CName":"**第一家","CNum":12345678},{"CIP":"10.255.137.200","CName":"**第二家","CNum":223456789}]

curl "http://127.0.0.1/qwe"
�cinemaIP��
10.255.126.34**第一家 1234567810.255.137.200**第二家 23456789

希望您能点醒梦中人！

drizzle_server settings and kqueue/kevent messages

I've been trying to tune the drizzle_server settings for performance, but I don't understand all of the error messages that are logged. Can you provide any information that might help?

Relevant config entries:
I set mysql's connection limit to 2000.

for events:
worker_connections 4096;

for upstream:
drizzle_server 127.0.0.1:3306 dbname=foo password=bar user=me protocol=mysql;
drizzle_keepalive max=2000 mode=single overflow=ignore;

There are the log entries I'm seeing:

"kqueue change list is filled up while connecting to upstream"
and
"kevent() reported that connect() failed (54: Connection reset by peer)"

I think the first means there are too many connections waiting for mysql? And the second means the connection to mysql was reset? Is it possible these are related to sysctl limits?

I am testing this on OSX, with ulimit -n set to 8192. I am using ab both with and without keep alive enabled and with 500 and 1000 connections.

Thanks.

Thanks.

drizzle openresty mysql5.5 not working

Hello...
I have Below Server configuration and not working drizzle mysql on 5.5 version.
So Is there any way to fix below issue ?

nginx version: openresty/1.9.3.1
mysql Ver 14.14 Distrib 5.5.46, for Linux (x86_64) using readline 5.1

2015/12/20 17:31:51 [error] 28614#0: *9 failed to connect: 10: drizzle_state_handshake_server_read:bad packet size:67:89 while connecting to drizzle upstream, client: 103.240.32.22, server: myserver.com, request: "GET /quote HTTP/1.1", upstream: "drizzle://127.0.0.1:3306", host: "myserver.com"

Thanks

drizzle_output directive support ( like postgres_output )

I would like to response single value from the result-set in text or binary format.
For example, response binary PNG data directly with content-type "image/png".

Please support "drizzle_output" like ngx_postgres.