ip2location / ip2proxy-nginx Goto Github PK

How can I detect the visitors who are using VPN and then return them to error 404?

Hi @ip2location ,

The setting of "ip2proxy_access_type shared_memory/cache_memory" in nginx.conf seems loading fail, and saw it go back to file_io.
Therefore, it cause cpu loading up to 100%, but did not show any error message.

ngx_http_ip2proxy.c:356
    IP2Proxy_open_mem(gcf->handler, gcf->access_type);

Kindly help me to check this issue. Thanks a lot.

Hi Support,

I download latest the ip2proxy-c and ip2proxy-nginx, and smoothly complied nginx 1.16.1.
but I can't start nginx 1.16.1 when enable ip2proxy.

Can help me check this status?

Here are some details to reproduce the problem:

root@test:~# /etc/sbin/nginx -v
nginx version: nginx/1.16.1
root@test:~# /etc/sbin/nginx -t
Segmentation fault (core dumped)
root@test:~# cat /etc/nginx/nginx.conf
load_module /etc/modules/ngx_http_ip2proxy_module.so;

worker_processes  1;
events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        on;
    keepalive_timeout  65;

    ip2proxy_database /home/ubuntu/IP2PROXY-IP-PROXYTYPE-COUNTRY.BIN;
    ip2proxy_access_type cache_memory;

    server {
        listen       80;
        server_name  localhost;

        location / {
            root   html;
            index  index.html index.htm;
        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

Found a segfault on line (void) ngx_copy((void *)ip_addr, addr.data, addr.len); when addr.data contains two IP addresses 2806:103e:21:7021:15e1:2a39:ed91:263b,66.249.80.67

Hello, i build nginx 1.20.2 and 1.22.1 with latest ip2proxy library, then my nginx start writes country codes to access.log via $ip2proxy_proxy_type variable.

My full build script of nginx+ip2plocation+ip2proxy and sources in attachment:

compile.tar.gz.zip

My log_format:

log_format main '{"domain": "$host", "time_local": "$time_local", "remote_addr": "$remote_addr", "realip_remote_addr": "$realip_remote_addr", "http_x_forwarded_for": "$http_x_forwarded_for", "http_cf_connecting_ip": "$http_cf_connecting_ip", "ip2location_country_short": "$ip2location_country_short", "ip2location_country_long": "$ip2location_country_long", "ip2proxy_proxy_type": "$ip2proxy_proxy_type", "status": $status, "upstream_status": "$upstream_status", "upstream_cache_status": "$upstream_cache_status", "upstream_addr": "$upstream_addr", "bytes_sent": "$bytes_sent", "msec": "$msec", "connection": "$connection", "connection_requests": "$connection_requests", "request_time": $request_time, "request_method": "$request_method", "request_uri": "$request_uri","request": "$request", "full_url": "$scheme://$host$request_uri", "http_referer": "$http_referer", "http_user_agent": "$http_user_agent", "remote_user": "$remote_user"}';

My Log message:

{"domain": "www.mysite.com", "time_local": "23/Mar/2023:00:06:54 -0400", "remote_addr": "66.102.8.29", "realip_remote_addr": "66.102.8.29", "http_x_forwarded_for": "104.253.196.232", "http_cf_connecting_ip": "-", "ip2location_country_short": "US", "ip2location_country_long": "United States of America", "ip2proxy_proxy_type": "US", "status": 444, "upstream_status": "-", "upstream_cache_status": "-", "upstream_addr": "-", "bytes_sent": "0", "msec": "1679544414.678", "connection": "7590019", "connection_requests": "1", "request_time": 0.000, "request_method": "GET", "request_uri": "/galleries","request": "GET /galleries HTTP/1.1", "full_url": "https://www.mysite.com/galleries/", "http_referer": "-", "http_user_agent": "Mozilla/5.0 (Windows NT 5.0) AppleWebKit/5340 (KHTML, like Gecko) Chrome/15.0.820.0 Safari/5340,gzip(gfe)", "remote_user": "-"}

Trying to compile the module with nginx 1.23 results in the following error:

/root/extracted/ip2proxy-nginx-master/ngx_http_ip2proxy.c: In function ‘ngx_http_ip2proxy_get_records’: /root/extracted/ip2proxy-nginx-master/ngx_http_ip2proxy.c:246:22: error: assignment to ‘ngx_array_t *’ from incompatible pointer type ‘ngx_table_elt_t **’ {aka ‘struct ngx_table_elt_s **’} [-Werror=incompatible-pointer-types] 246 | xfwd = &r->headers_in.x_forwarded_for; | ^ /root/extracted/ip2proxy-nginx-master/ngx_http_ip2proxy.c:249:70: error: passing argument 3 of ‘ngx_http_get_forwarded_addr’ from incompatible pointer type [-Werror=incompatible-pointer-types] 249 | (void) ngx_http_get_forwarded_addr(r, &addr, xfwd, NULL, gcf->proxies, gcf->proxy_recursive); | ^~~~ | | | ngx_array_t * In file included from src/http/ngx_http.h:36, from /root/extracted/ip2proxy-nginx-master/ngx_http_ip2proxy.c:10: src/http/ngx_http_core_module.h:532:22: note: expected ‘ngx_table_elt_t *’ {aka ‘struct ngx_table_elt_s *’} but argument is of type ‘ngx_array_t *’ 532 | ngx_table_elt_t *headers, ngx_str_t *value, ngx_array_t *proxies, | ~~~~~~~~~~~~~~~~~^~~~~~~

it is necessary that the module determines the proxy not from the IP of the real source (remote addr or x-forwarder-for), but from the GET parameter ip (enable to configure in the config)

Hello, in latest version You are forget " double quotes in line 178:15.

Affect: ngx_string("ip2proxy_domain)

Please fix.

What is the best way to update (or replace) a DB.BIN database on nginx?

Hello, can`t compile nginx 1.18.0 with ip2proxy module.

Upd: 1.16.1 too can`t compile :(

Error:

        -o objs/addon/ip2mod-proxy/ngx_http_ip2proxy.o \
        /root/compile/ip2mod-proxy/ngx_http_ip2proxy.c
cc -c -pipe  -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -g -O2 -fdebug-prefix-map=/root/compile/nginx-1.18.0=. -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC -I src/core -I src/event -I src/event/modules -I src/os/unix -I objs \
        -o objs/ngx_modules.o \
        objs/ngx_modules.c
/root/compile/ip2mod-proxy/ngx_http_ip2proxy.c: In function ‘ngx_http_ip2proxy_exit_process’:
/root/compile/ip2mod-proxy/ngx_http_ip2proxy.c:301:4: error: implicit declaration of function ‘IP2Proxy_DB_del_shm’ [-Werror=implicit-function-declaration]
    IP2Proxy_DB_del_shm();
    ^~~~~~~~~~~~~~~~~~~
sed -e "s|%%PREFIX%%|/etc/nginx|" \
        -e "s|%%PID_PATH%%|/var/run/nginx.pid|" \
        -e "s|%%CONF_PATH%%|/etc/nginx/nginx.conf|" \
        -e "s|%%ERROR_LOG_PATH%%|/var/log/nginx/error.log|" \
        < man/nginx.8 > objs/nginx.8
cc1: all warnings being treated as errors
objs/Makefile:1827: recipe for target 'objs/addon/ip2mod-proxy/ngx_http_ip2proxy.o' failed
make[2]: *** [objs/addon/ip2mod-proxy/ngx_http_ip2proxy.o] Error 1
make[2]: Leaving directory '/root/compile/nginx-1.18.0/debian/build-nginx'
Makefile:8: recipe for target 'build' failed
make[1]: *** [build] Error 2
make[1]: Leaving directory '/root/compile/nginx-1.18.0/debian/build-nginx'
debian/rules:50: recipe for target 'build-arch.nginx' failed
make: *** [build-arch.nginx] Error 2
dpkg-buildpackage: error: debian/rules build gave error exit status 2

CFLAGS:

CFLAGS="" ./configure --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --add-module=/root/compile/ip2mod-location --add-module=/root/compile/ip2mod-proxy --with-cc-opt="-g -O2 -fdebug-prefix-map=/root/compile/nginx-1.18.0=. -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC" --with-ld-opt="-Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie"

GCC Version:

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/6/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 6.3.0-18+deb9u1' --with-bugurl=file:///usr/share/doc/gcc-6/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-6 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-6-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-6-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-6-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1)```

I compiled it successfully. However, when I added it to the nginx. I got the error.

ip2proxy on;
ip2proxy_database /etc/ip2proxy/IP2PROXY-LITE-PX10.BIN;
ip2proxy_reverse_proxy on;
ip2proxy_access_type shared_memory;

nginx: [emerg] unknown directive "ip2proxy" in /etc/nginx/nginx.conf:20

Hello @ip2location ,

In may 2019 I opened a first issue on this topic, would it be possible to add a method to the module to reload/refresh the file when using cache_memory ?

it could be located in the location context, something like :

location /refresh_proxy {

ip2proxy_refresh;

}

Where ip2proxy_refresh would clean the memory and reload the file in memory.

Is there any ETA for such feature ?

Thanks

Hello, on middle loaded nginx 1.18+ server (2000-4000+ rps and proxy_pass to http backends) with compiled ip2location and ip2proxy:

I checked 1.18.0 and 1.20.1 versions of nginx with this bugs (nginx flags same on 1.16.1, 1.18.0, 1.20.1).

OS: Debian 9 x86_64 Stretch
Compiled nginx flags:

nginx version: nginx/1.18.0
built by gcc 6.3.0 20170516 (Debian 6.3.0-18+deb9u1)
built with OpenSSL 1.1.1d 10 Sep 2019
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --add-module=/root/compile/ip2mod-location --add-module=/root/compile/ip2mod-proxy --with-cc-opt='-g -O2 -fdebug-prefix-map=/root/compile/nginx-1.18.0=. -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'

I use latest libraries for building ip2location/ip2proxy with nginx:

https://github.com/chrislim2888/IP2Location-C-Library/archive/master.zip
https://github.com/ip2location/ip2proxy-c/archive/master.zip
https://github.com/ip2location/ip2location-nginx/archive/master.zip
https://github.com/ip2location/ip2proxy-nginx/archive/master.zip

Dmesg errors:

[Mon Oct 25 05:25:21 2021] show_signal_msg: 66 callbacks suppressed
[Mon Oct 25 05:25:21 2021] nginx[12799]: segfault at 55ef2ac5d359 ip 00007f9cae118c77 sp 00007ffdcd1c3800 error 4 in libIP2Proxy.so.2.0.0[7f9cae115000+5000]
[Mon Oct 25 05:25:21 2021] nginx[12831]: segfault at 55ef2ac5d37d ip 00007f9cae118c77 sp 00007ffdcd1c3800 error 4 in libIP2Proxy.so.2.0.0[7f9cae115000+5000]
[Mon Oct 25 05:25:21 2021] nginx[12814]: segfault at 55ef2ac5d331 ip 00007f9cae118c77 sp 00007ffdcd1c37c0 error 4 in libIP2Proxy.so.2.0.0[7f9cae115000+5000]
[Mon Oct 25 05:25:21 2021] traps: nginx[12821] trap invalid opcode ip:7f9cae118c76 sp:7ffdcd1c3800 error:0
[Mon Oct 25 05:25:21 2021] in libIP2Proxy.so.2.0.0[7f9cae115000+5000]
[Mon Oct 25 05:25:21 2021] nginx[12838]: segfault at 55ef2ac5d3f0 ip 00007f9cae118c77 sp 00007ffdcd1c3800 error 4 in libIP2Proxy.so.2.0.0[7f9cae115000+5000]
[Mon Oct 25 05:25:21 2021] traps: nginx[12859] trap invalid opcode ip:7f9cae118c76 sp:7ffdcd1c3800 error:0
[Mon Oct 25 05:25:21 2021] in libIP2Proxy.so.2.0.0[7f9cae115000+5000]
[Mon Oct 25 05:25:21 2021] nginx[12864]: segfault at 55ef2ac5d3b0 ip 00007f9cae118c77 sp 00007ffdcd1c3800 error 4 in libIP2Proxy.so.2.0.0[7f9cae115000+5000]
[Mon Oct 25 05:25:21 2021] nginx[12884]: segfault at 55ef2ac5d335 ip 00007f9cae118c77 sp 00007ffdcd1c3800 error 4 in libIP2Proxy.so.2.0.0[7f9cae115000+5000]
[Mon Oct 25 05:25:21 2021] nginx[12883]: segfault at 55ef2ac5d37b ip 00007f9cae118c77 sp 00007ffdcd1c3800 error 4 in libIP2Proxy.so.2.0.0[7f9cae115000+5000]
[Mon Oct 25 05:25:21 2021] nginx[12905]: segfault at 55ef2ac5d3d8 ip 00007f9cae118c77 sp 00007ffdcd1c3800 error 4 in libIP2Proxy.so.2.0.0[7f9cae115000+5000]

And this is not all, also have a strange trouble when nginx workers consume 100% with system usage(in the top), and in error logs of nginx in this moment i have a tons of messages like this:

2021/10/28 22:41:06 [error] 34416#34416: *51265 upstream prematurely closed connection while reading response header from upstream.

But upstream is ok - this is very quick upstream written on Go language based on Iris and in case of nginx 1.16.1 no have any problems.

strace connected to any 100% cpu usage nginx worker shows
lseek method in infinity loop to only one descriptor like this:

lseek(1129, 70376488, SEEK_SET) = 70376488
lseek(1129, 70376488, SEEK_SET) = 70376488
lseek(1129, 70376488, SEEK_SET) = 70376488
lseek(1129, 70376488, SEEK_SET) = 70376488
lseek(1129, 70376488, SEEK_SET) = 70376488
lseek(1129, 70376488, SEEK_SET) = 70376488
lseek(1129, 70376488, SEEK_SET) = 70376488

But with 1.16.1 version no have any of this problems.

But I cannot use more newer nginx with ip2location and ip2proxy library :(

I don`t know ip2location library have any problems or not with nginx 1.18+, but ip2proxy have.

Hi @ip2location ,
I have met some issue.
I use the lastest commit and did not used the ip2proxy_access_type.

Test setting:

nginx >> worker_processes 1;
initial >> work fine
reload multi times>> work fine

nginx >> worker_processes 8;
initial >> work fine
reload one time >> 1 of 8 process up to 100%
reload second time >> 2 of 8 process up to 100%

The process which cpu up to 100% seems goes to file_io, in this situation, the proxy_Type can not get value.
Didn't know if this situation is database info didn't write in memory.
Please help to check, thank you.