GithubHelp home page GithubHelp logo

mysql-udf-http's Introduction

MySQL User-defined function (UDF) for HTTP GET/POST

MySQL User-defined function (UDF) for HTTP REST

Note: It is a fork repository. Original Website is below.
http://code.google.com/p/mysql-udf-http

Overview

HTTP Method CRUD Action Description
POST CREATE Create a new resource
GET READ Read a resource
PUT UPDATE Update a resource
DELETE DELETE Delete a resource

Support MySQL version 5.1.x and 5.5.x on Linux systems.

User Guide

0. Prepare

make sure these depandencies are installed.

  • mysql server
  • mysql_config command
  • development tools

For Debian, like this.

apt-get install mysql-server
apt-get install make
apt-get install gcc
apt-get install libmysqlclient-dev
apt-get install pkg-config

1. Install on Linux

This sample is written for "/usr/local/webserver/mysql/" is your MySQL install path.

ulimit -SHn 65535
wget http://curl.haxx.se/download/curl-7.21.1.tar.gz
tar zxvf curl-7.21.1.tar.gz
cd curl-7.21.1/
./configure --prefix=/usr
make && make install
cd ../

echo "/usr/local/webserver/mysql/lib/mysql/" > /etc/ld.so.conf.d/mysql.conf
/sbin/ldconfig
wget http://mysql-udf-http.googlecode.com/files/mysql-udf-http-1.0.tar.gz
tar zxvf mysql-udf-http-1.0.tar.gz
cd mysql-udf-http-1.0/
./configure --prefix=/usr/local/webserver/mysql --with-mysql=/usr/local/webserver/mysql/bin/mysql_config
make && make install
cd ../

2. Enter to the MySQL console

/usr/local/webserver/mysql/bin/mysql -S /tmp/mysql.sock

3. Create the UDF function in the MySQL console

mysql>

create function http_get returns string soname 'mysql-udf-http.so';
create function http_post returns string soname 'mysql-udf-http.so';
create function http_put returns string soname 'mysql-udf-http.so';
create function http_delete returns string soname 'mysql-udf-http.so';

4. Usage

Description:

mysql>
SELECT http_get('<url>');
SELECT http_post('<url>', '<data>');
SELECT http_put('<url>', '<data>');
SELECT http_delete('<url>');

Example (A):

mysql>

/* Baidu Mobile Search */
SELECT http_get('http://m.baidu.com/s?word=xoyo&pn=0');
SELECT http_post('http://m.baidu.com/s','word=xoyo&pn=0');

/* Sina Weibo Open Platform */
SELECT http_get('http://api.t.sina.com.cn/statuses/user_timeline/103500.json?count=1&source=1561596835') AS data;
SELECT http_post('http://your_sina_uid:[email protected]/statuses/update.xml?source=1561596835', 'status=Thins is sina weibo test information');

/* Tokyo Tyrant */
SELECT http_put('http://192.168.8.34:1978/key', 'This is value');
SELECT http_get('http://192.168.8.34:1978/key');
SELECT http_delete('http://192.168.8.34:1978/key');

Example (B):

Use mysql-udf-http and lib_mysqludf_json synchronizes the update to Tokyo Tyrant with MySQL Trigger.

(1). Download and install lib_mysqludf_json:

On 32-Bit Linux:

wget http://mysql-udf-http.googlecode.com/files/lib_mysqludf_json-i386.tar.gz
tar zxvf lib_mysqludf_json-i386.tar.gz
cd lib_mysqludf_json-i386/
# if your MySQL install path is not '/usr/local/webserver/mysql/', please modify the path.
cp -f lib_mysqludf_json.so /usr/local/webserver/mysql/lib/mysql/plugin/lib_mysqludf_json.so
cd ../

On 64-Bit Linux:

wget http://mysql-udf-http.googlecode.com/files/lib_mysqludf_json-x86_64.tar.gz
tar zxvf lib_mysqludf_json-x86_64.tar.gz
cd lib_mysqludf_json-x86_64/
# if your MySQL install path is not '/usr/local/webserver/mysql/', please modify the path.
cp -f lib_mysqludf_json.so /usr/local/webserver/mysql/lib/mysql/plugin/lib_mysqludf_json.so
cd ../

Enter to the MySQL console:

/usr/local/webserver/mysql/bin/mysql -S /tmp/mysql.sock
mysql>

create function lib_mysqludf_json_info returns string soname 'lib_mysqludf_json.so';
create function json_array returns string soname 'lib_mysqludf_json.so';
create function json_members returns string soname 'lib_mysqludf_json.so';
create function json_object returns string soname 'lib_mysqludf_json.so';
create function json_values returns string soname 'lib_mysqludf_json.so';

How to use lib_mysqludf_json, please visit: http://www.mysqludf.org/lib_mysqludf_json/

(2). Create table:

mysql>

SET NAMES UTF8;
USE test;
CREATE TABLE IF NOT EXISTS `mytable` (
  `id` int(10) NOT NULL AUTO_INCREMENT,
  `addtime` int(10) NOT NULL,
  `title` varchar(255) CHARACTER SET utf8 NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 AUTO_INCREMENT=1;
(3). Create trigger for table:

mysql>

/* TRIGGER for INSERT */
DELIMITER |
DROP TRIGGER IF EXISTS mytable_insert;
CREATE TRIGGER mytable_insert
AFTER INSERT ON mytable
FOR EACH ROW BEGIN
    SET @tt_json = (SELECT json_object(id,addtime,title) FROM mytable WHERE id = NEW.id LIMIT 1);
    SET @tt_resu = (SELECT http_put(CONCAT('http://192.168.8.34:1978/', NEW.id), @tt_json));
END |
DELIMITER ;

/* TRIGGER for UPDATE */
DELIMITER |
DROP TRIGGER IF EXISTS mytable_update;
CREATE TRIGGER mytable_update
AFTER UPDATE ON mytable
FOR EACH ROW BEGIN
    SET @tt_json = (SELECT json_object(id,addtime,title) FROM mytable WHERE id = OLD.id LIMIT 1);
    SET @tt_resu = (SELECT http_put(CONCAT('http://192.168.8.34:1978/', OLD.id), @tt_json));
END |
DELIMITER ;

/* TRIGGER for DELETE */
DELIMITER |
DROP TRIGGER IF EXISTS mytable_delete;
CREATE TRIGGER mytable_delete
AFTER DELETE ON mytable
FOR EACH ROW BEGIN
    SET @tt_resu = (SELECT http_delete(CONCAT('http://192.168.8.34:1978/', OLD.id)));
END |
DELIMITER ;

(4). Select data from MySQL table and Tokyo Tyrant:

mysql>

SELECT id,addtime,title,http_get(CONCAT('http://192.168.8.34:1978/',id)) AS tt FROM mytable ORDER BY id DESC LIMIT 0,5;

5. How to drop the UDF function

mysql>

drop function http_get;
drop function http_post;
drop function http_put;
drop function http_delete;

Author

Please check the Members at the left side of menu. https://code.google.com/p/mysql-udf-http/

License

This is free software, and you are welcome to modify and redistribute it under the New BSD License.

mysql-udf-http's People

Contributors

y-ken avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

mysql-udf-http's Issues

Invalid free() / delete / delete[] / realloc()

Problem
---------

Compiled UDF and loaded into 5.7.17.

==12629== Invalid free() / delete / delete[] / realloc()
==12629==    at 0x4C2AD17: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12629==    by 0x29566419: http_delete_deinit (mysql-udf-http.c:338)
==12629==    by 0x85CF82: udf_handler::cleanup() (item_func.cc:4363)
==12629==    by 0x85D027: Item_udf_func::cleanup() (item_func.cc:4686)
==12629==    by 0xCAC5AF: delete_self (item.h:2122)
==12629==    by 0xCAC5AF: Query_arena::free_items() (sql_class.cc:3391)
==12629==    by 0xCB1306: THD::cleanup_after_query() (sql_class.cc:2303)
==12629==    by 0xCF91D5: mysql_parse(THD*, Parser_state*) (sql_parse.cc:5634)
==12629==    by 0xCFA627: dispatch_command(THD*, COM_DATA const*, enum_server_command) (sql_parse.cc:1461)
==12629==    by 0xCFB4C3: do_command(THD*) (sql_parse.cc:999)
==12629==    by 0xDC858B: handle_connection (connection_handler_per_thread.cc:300)
==12629==    by 0xF7DF43: pfs_spawn_thread (pfs.cc:2188)
==12629==    by 0x4E3CDF4: start_thread (pthread_create.c:308)
==12629==    by 0x61AD60C: clone (clone.S:113)
==12629==  Address 0xabababababababab is not stack'd, malloc'd or (recently) free'd

How to repeat
----------------

Launch mysqld in valgrind. Run the following SQL;

select http_delete(st_polygonfromtext(1));

Can't find symbol '_mysql_plugin_interface_version_' in library

MariaDB [(none)]> install soname 'mysql-udf-http';
ERROR 1127 (HY000): Can't find symbol 'mysql_plugin_interface_version' in library
MariaDB [(none)]>

OS: Ubuntu 16.04 LTS

ubuntu@ip-172-31-3-190:/usr/lib/mysql/plugin$ ls -l
...
...
...
-rwxr-xr-x 1 root root 897 Aug 20 19:35 mysql-udf-http.la
lrwxrwxrwx 1 root root 23 Aug 20 19:35 mysql-udf-http.so -> mysql-udf-http.so.0.0.0
lrwxrwxrwx 1 root root 23 Aug 20 19:35 mysql-udf-http.so.0 -> mysql-udf-http.so.0.0.0
-rwxr-xr-x 1 root root 18736 Aug 20 19:35 mysql-udf-http.so.0.0.0
...
...
...

Crash on http_post

i got multiple mysqld crashes that have the mysql-udf-http module in its trace.

i have multple triggers that use the http_post function to send json data. but not sure which circumstances lead to the crash, i just want to document it.
is it possible that there is a post data size that could lead to the crash? in my case, i possible post multiple MB of data.

 161208 11:37:50 [ERROR] mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see http://kb.askmonty.org/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.

Server version: 10.0.17-MariaDB-1~wheezy-wsrep
key_buffer_size=209715200
read_buffer_size=4194304
max_used_connections=495
max_threads=502
thread_count=45
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 4327763 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0x7f83750cf008
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7f85795ffe30 thread_stack 0x48000
/usr/sbin/mysqld(my_print_stacktrace+0x2b)[0x55a5566a626b]
/usr/sbin/mysqld(handle_fatal_signal+0x422)[0x55a55622c072]
/lib/x86_64-linux-gnu/libpthread.so.0(+0xf8d0)[0x7f93b94e28d0]
/usr/sbin/mysqld(free+0x63)[0x55a556707ac3]
/usr/lib/mysql/plugin/mysql-udf-http.so(http_post_deinit+0x12)[0x7f92b1dfd052]
/usr/sbin/mysqld(_ZN11udf_handler7cleanupEv+0x39)[0x55a5562868b9]
/usr/sbin/mysqld(_ZN13Item_udf_func7cleanupEv+0x18)[0x55a556286968]
/usr/sbin/mysqld(_Z13cleanup_itemsP4Item+0x21)[0x55a5560b0c71]
/usr/sbin/mysqld(_ZN7sp_head7executeEP3THDb+0x734)[0x55a55632b414]
/usr/sbin/mysqld(_ZN7sp_head15execute_triggerEP3THDPK19st_mysql_lex_stringS4_P13st_grant_info+0x19b)[0x55a55632b7fb]
/usr/sbin/mysqld(_ZN19Table_triggers_list16process_triggersEP3THD14trg_event_type20trg_action_time_typeb+0x125)[0x55a5561471c5]
/usr/sbin/mysqld(_Z12mysql_updateP3THDP10TABLE_LISTR4ListI4ItemES6_PS4_jP8st_ordery15enum_duplicatesbPySB_+0x11eb)[0x55a55615207b]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x2c0b)[0x55a5560b95ab]
/usr/sbin/mysqld(+0x43e60a)[0x55a5560bf60a]
/usr/sbin/mysqld(+0x43ef67)[0x55a5560bff67]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x18a2)[0x55a5560c1e02]
/usr/sbin/mysqld(_Z10do_commandP3THD+0x21f)[0x55a5560c272f]
/usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x58b)[0x55a55618449b]
/usr/sbin/mysqld(handle_one_connection+0x47)[0x55a556184597]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x80a4)[0x7f93b94db0a4]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f93b7a6307d]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.