undefined reference to `g_get_monotonic_time'

When a client issue an INITDB command this applies only to the master. Also the slave needs to be reconfigured

Currently, all the COM_QUERY packets are routed according to a combination of rules that define username, schemaname, query pattern and more.
This means that only queries have a configurable destination, while all other packets are always sent to hostgroup 0.

Enhancement:
should be possible to route any packet that is not a COM_QUERY using rules based on username/schemaname

Chan provided some code that is able to tokenize the queries. That code needs to be merged and then enhanced

ProxySQL doesn't correctly handle connections terminated by the mysql backends.
The logic behind mysql_auto_reconnect_enabled and reconnect_server_on_shut_fd() needs to be reviewed .

The only workaround right now is to have a small value for mysql_wait_timeout

ProxySQL currently bind on any interface.

Should be possible to configure it to bind an any interface or localhost only.

I had to merge mysql_protocol changes from his fork manually otherwise proxysql wouldn't build on my scientific linux box with mariadb 5.5 libs and dev

This should prevent daemonize

Data stream larger than 2GB causes proxysql to hang.

Tokenizer should log into SQLite internal DB, and its behaviour should be tunable via configuration file.

Tasks performed by qr_report_thread() should be moved into admin thread.

Possible configurable behaviours:

• enable/disable switch
• frequency of dumps
• save output as aggregate or time series

mysql_hostgroups=N needs to be configured under [mysql] section in proxysql.cnf . It defines the number of hostgroups.
All the defined hostgroup that are >= mysql_hostgroups-1 are ignored .

TODO:
a) document mysql_hostgroups
b) increase the default value (currently is 2 only)
c) generate errors when are found references to not existing hostgroups

Only one resultset is currently handled. Multiple resulteset are used when calling stored procedure

I need to document:

• how to add/remove/enable/disable users
• how to reload user table

ProxySQL doesn't track client's source (hostname, ip, etc)

Should be possible to drop the whole query cache.

I need to document how to enable/disable debugging, and how to tune what to log.

l_alloc and l_free implement a per-thread memory allocator where memory buffers are referenced in stacks.
If a specific workload allocates too much memory blocks in a stack and then release them, they will remain unused.
To avoid memory spikes, we need to implement a garbage collector for l_alloc/l_free

To prevent data lost in case of failure during sync to disk, this operation needs to be performed in a transaction

When a data stream is closed, net buffers associated with it aren't deallocated but marked as free and available for other data stream.
This is an architectural decision to speed up the creation of new connections.
Although, it has the drawback that a connections spike can allocate a lot of buffers and these will never be released again.

Some optimizations can be applied, like allocating not more a fixed amount of buffers, or implement a sort of garbage collector to free unnecessary memory.

Hi Rene,

I'm trying to compile proxysql on a centos 6.5 machine, but I had no luck.
make prints several errors like
mysql_protocol.c:7: warning: pointer targets in passing argument 1 of ‘strlen’ differ in signedness
/usr/include/string.h:399: note: expected ‘const char *’ but argument is of type ‘unsigned char *’

All errors say expected ‘const char *’ but argument is of type ‘unsigned char *’

Then finally
mysql_protocol.c: At top level:
mysql_protocol.c:273: warning: ‘struct rand_struct’ declared inside parameter list
mysql_protocol.c:273: error: conflicting types for ‘proxy_create_random_string’
../include/mysql_protocol.h:22: note: previous declaration of ‘proxy_create_random_string’ was here
mysql_protocol.c: In function ‘proxy_create_random_string’:
mysql_protocol.c:276: warning: passing argument 1 of ‘proxy_my_rnd’ from incompatible pointer type
../include/mysql_protocol.h:23: note: expected ‘struct rand_struct ’ but argument is of type ‘struct rand_struct *’
mysql_protocol.c: At top level:
mysql_protocol.c:282: warning: ‘struct rand_struct’ declared inside parameter list
mysql_protocol.c:282: error: conflicting types for ‘proxy_my_rnd’
../include/mysql_protocol.h:23: note: previous declaration of ‘proxy_my_rnd’ was here
mysql_protocol.c: In function ‘proxy_my_rnd’:
mysql_protocol.c:283: error: dereferencing pointer to incomplete type
mysql_protocol.c:283: error: dereferencing pointer to incomplete type
mysql_protocol.c:283: error: dereferencing pointer to incomplete type
mysql_protocol.c:283: error: dereferencing pointer to incomplete type
mysql_protocol.c:284: error: dereferencing pointer to incomplete type
mysql_protocol.c:284: error: dereferencing pointer to incomplete type
mysql_protocol.c:284: error: dereferencing pointer to incomplete type
mysql_protocol.c:284: error: dereferencing pointer to incomplete type
mysql_protocol.c:285: error: dereferencing pointer to incomplete type
mysql_protocol.c:285: error: dereferencing pointer to incomplete type
make: ** [obj/mysql_protocol.o] Error 1

I have installed glib2c-devel, openssl-devel, gcc, pcre-devel and glib-devel.

Am I missing something?

Thanks

Leni

ProxySQL uses an internal database to store configuration and other metadata in binary format on a file.
The path for such file needs to be configurable

How to reproduce it:

$ mysql -u root -h 127.0.0.1 -P6033
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3073370944
Server version: 5.1.30 (Ubuntu)

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> \s

mysql Ver 14.14 Distrib 5.5.35, for debian-linux-gnu (i686) using readline 6.2

Connection id: 3073370944
Current database: information_schema
Current user: root@localhost
SSL: Not in use
Current pager: stdout
Using outfile: ''
Using delimiter: ;
Server version: 5.1.30 (Ubuntu)
Protocol version: 10
Connection: 127.0.0.1 via TCP/IP
Server characterset: utf8
Db characterset: utf8
Client characterset: utf8
Conn. characterset: utf8
TCP port: 6033
Uptime: 12 days 10 hours 36 min 13 sec

Threads: 2 Questions: 2346446 Slow queries: 0 Opens: 735 Flush tables: 1 Open tables: 182 Queries per second avg: 2.182

mysql> \s

mysql Ver 14.14 Distrib 5.5.35, for debian-linux-gnu (i686) using readline 6.2

Connection id: 3073370944
Current database: information_schema
Current user: root@localhost
SSL: Not in use
Current pager: stdout
Using outfile: ''
Using delimiter: ;
Server version: 5.1.30 (Ubuntu)
Protocol version: 10
Connection: 127.0.0.1 via TCP/IP
Server characterset: utf8
Db characterset: utf8
Client characterset: utf8
Conn. characterset: utf8
TCP port: 6033

In error log:
Error: signal 11:
./proxysql(crash_handler+0x19)[0x8103ed9]
[0xb7743400]
./proxysql[0x80fee37]
./proxysql[0x80ff1c5]
./proxysql[0x810019b]
./proxysql(mysql_thread+0x91e)[0x8102b3c]
/lib/i386-linux-gnu/libpthread.so.0(+0x6d4c)[0xb76b5d4c]
/lib/i386-linux-gnu/libc.so.6(clone+0x5e)[0xb74d8bae]

a) Generate a warning on "localhost"
b) handle host specified in the form ip:port
c) rename the current proxysql.cnf in proxysql.cnf.sample
d) verify that connection parameters work

Configuration is currently saved on disk running "CONFIG SYNC TO DISK" .
Should be possible to automatically save on disk after every successful FLUSH command

If the proxysql.cnf file is not found in the current directory it should fallback to looking in /etc

This is expected behavior for most programs.

ProxySQL has CLIENT_MULTI_STATEMENTS disabled, but some clients seem to ignore this (for example, Connector/Net) and assume that CLIENT_MULTI_STATEMENTS is always enabled.
If a client uses such feature even if disabled, it will receive an SQL syntax error starting from the first " ; " .

We need to set MYSQL_OPT_CONNECT_TIMEOUT calling mysql_options() before calling mysql_real_connect()

In case a connection to the backend doesn't succeed, the client get disconnected

Reduce the number of connections to the database multiplexing multiple clients connections into fewer database connections

Well the title says all :)

Due the various dependencies, ProxySQL is at time uneasy to compile and deploy.
Binaries and/or packages for various distros need to be available

When a new node is added should be possible to rebalance the already established connections from the old nodes to use also the new node.

Issue #19 solved the problem of running large queries that return more than 2GB of data. We have run into a new issue. After running our query which pulls 4gb of data, the data is sorted on the client machine, then a new query is sent to proxysql, but the client is no longer able to connect to proxysql. I verified that the proxysql process is still running on the proxy machine using:

pgrep proxysql | xargs ps
6840 pts/0 Sl+ 0:43 ./proxysql

I then stop the indexing process on the client machine, but can no longer connect to the proxy. The only error the client receives is: Lost connection to MySQL server during query.

If I kill the proxysql process on the proxy machine and restart it, I am able to connect again from the client machine (the above bug happens each time though)

Following the directions i nthe readme, I get to this step:

mkdir ProxySQL
cd ProxySQL
wget https://downloads.mariadb.org/interstitial/mariadb-native-client/Source/mariadb-native-client.tar.gz
tar -zxf mariadb-native-client.tar.gz
cd mariadb-native-client
cmake . && make

Which fails:

[root@sph20 mariadb-native-client]# cmake . && make
-- Configuring done
-- Generating done
-- Build files have been written to: /ProxySQL/mariadb-native-client
Linking C shared library libmariadb.so
CMakeFiles/libmariadb.dir/my_secure.c.o: In function my_SSL_error': /ProxySQL/mariadb-native-client/libmysql/my_secure.c:43: undefined reference toERR_get_error'
/ProxySQL/mariadb-native-client/libmysql/my_secure.c:54: undefined reference to ERR_reason_error_string' CMakeFiles/libmariadb.dir/my_secure.c.o: In functionmy_ssl_end':
/ProxySQL/mariadb-native-client/libmysql/my_secure.c:174: undefined reference to CRYPTO_set_locking_callback' /ProxySQL/mariadb-native-client/libmysql/my_secure.c:175: undefined reference toCRYPTO_set_id_callback'
/ProxySQL/mariadb-native-client/libmysql/my_secure.c:177: undefined reference to CRYPTO_num_locks' /ProxySQL/mariadb-native-client/libmysql/my_secure.c:187: undefined reference toERR_free_strings'
/ProxySQL/mariadb-native-client/libmysql/my_secure.c:188: undefined reference to EVP_cleanup' /ProxySQL/mariadb-native-client/libmysql/my_secure.c:189: undefined reference toCONF_modules_unload'
/ProxySQL/mariadb-native-client/libmysql/my_secure.c:190: undefined reference to CRYPTO_cleanup_all_ex_data' CMakeFiles/libmariadb.dir/my_secure.c.o: In functionmy_ssl_start':
/ProxySQL/mariadb-native-client/libmysql/my_secure.c:111: undefined reference to CRYPTO_num_locks' /ProxySQL/mariadb-native-client/libmysql/my_secure.c:120: undefined reference toCRYPTO_num_locks'
/ProxySQL/mariadb-native-client/libmysql/my_secure.c:122: undefined reference to CRYPTO_set_id_callback' /ProxySQL/mariadb-native-client/libmysql/my_secure.c:123: undefined reference toCRYPTO_set_locking_callback'
/ProxySQL/mariadb-native-client/libmysql/my_secure.c:127: undefined reference to OPENSSL_config' /ProxySQL/mariadb-native-client/libmysql/my_secure.c:135: undefined reference toOPENSSL_add_all_algorithms_noconf'
collect2: ld returned 1 exit status
make[2]: *** [libmysql/libmariadb.so.1] Error 1
make[1]: *** [libmysql/CMakeFiles/libmariadb.dir/all] Error 2
make: *** [all] Error 2
[root@sph20 mariadb-native-client]#

This is on Centos 6.5

The crash is caused by any code calling proxy_debug() before glo_debug is initialized

Debugging messages are currently written in the error log.
A debug log should be used instead.

Only a subset of authentication capabilities are handled correctly.
Use http://dev.mysql.com/doc/internals/en/connection-phase-packets.html to extend the capabilities of check_client_authentication_packet()

If any server is deleted from the servers or hostgroups table while in ONLINE state, the server is still being used in active connections.
This behaviour is perhaps not what users expect.

FLUSH HOSTGROUPS should validate the content of "servers" and "hostgroups" against its internal data and if any ONLINE server is missing should return an error

Connections are recycled when connection pooling is enabled: this causes issues with prepared statements.

Possible solutions:

• disable connection pooling when prepared statements are detected
• reset connection when it is recycled

ProxySQL always tries to open and parse proxysql.cnf even if a not default config file is specified on command line, ignoring this one.

COM_CHANGE_USER is currently not supported

Using a GLib older than 2.32 will result in the follow error:

GLib-ERROR **: The thread system is not yet initialized.
aborting...
Aborted (core dumped)

Trying to connect to a mysql server with old password authentication generates the error:
Can't connect twice. Already connected

This is caused by this bug in MariaDB client library:
https://mariadb.atlassian.net/browse/CONC-1

Under certain circumstances (TBD), connections are recycled and ProxySQL continues creating new connections to the backends.
These circumstances need to be identified.

I didn't see this in the tasks, but wanted to add it. For a production environment, we will of course want to run proxysql in the background as a daemon. This allows us to setup things like monit to automatically monitor and restart the service should anything go wrong.

Query cache uses the hash of the query as key. This can lead to conflicts if the same query is executed in two different schemas or from two different users (potentially can also be a security issue). The keys n the query cache should also include schema and username.

setsockopt() with SO_REUSEADDR needs to be called before bind() to take effect