sirithink / mysql_binlog

Library for parsing MySQL binary logs in Ruby

Library for parsing MySQL binary logs in Ruby

This library parses a MySQL binary log in pure Ruby and produces hashes as output, much like the following Query event:

{:type=>:query_event,
 :position=>107,
 :filename=>"mysql-bin.000001",
 :header=>
  {:event_type=>2,
   :server_id=>1,
   :flags=>[],
   :event_length=>117,
   :timestamp=>1340414127,
   :next_position=>224},
 :event=>
  {:thread_id=>1,
   :query=>"create table a (id int, a char(100), primary key (id))",
   :status=>
    {:sql_mode=>0,
     :charset=>
      {:character_set_client=>
        {:character_set=>:utf8, :collation=>:utf8_general_ci},
       :collation_connection=>
        {:character_set=>:utf8, :collation=>:utf8_general_ci},
       :collation_server=>
        {:character_set=>:latin1, :collation=>:latin1_swedish_ci}},
     :flags2=>[],
     :catalog=>"std"},
   :elapsed_time=>0,
   :error_code=>0,
   :db=>"test"}}

Status

All event types can be read, but may not be parsed, as not all event types are currently fully supported. Over time this will improve. The current status of event support is documented below.

Event Types

ID	Event Type	Status
1	start_event_v3	Unsupported (deprecated).
2	query_event	Fully supported with all fields parsed.
3	stop_event	Fully supported with all fields parsed.
4	rotate_event	Fully supported with all fields parsed.
5	intvar_event	Fully supported with all fields parsed.
6	load_event	Unsupported (deprecated).
7	slave_event	Unsupported (deprecated).
8	create_file_event	Unsupported (deprecated).
9	append_block_event	Unsupported.
10	exec_load_event	Unsupported (deprecated).
11	delete_file_event	Unsupported.
12	new_load_event	Unsupported (deprecated).
13	rand_event	Fully supported with all fields parsed.
14	user_var_event	Fully supported with all fields parsed.
15	format_description_event	Fully supported with all fields parsed.
16	xid_event	Fully supported with all fields parsed.
17	begin_load_query_event	Unsupported.
18	execute_load_query_event	Unsupported.
19	table_map_event	Fully supported with all fields parsed.
20	pre_ga_write_rows_event	Unsupported (deprecated).
21	pre_ga_update_rows_event	Unsupported (deprecated).
22	pre_ga_delete_rows_event	Unsupported (deprecated).
23	write_rows_event	Fully supported with all fields parsed.
24	update_rows_event	Fully supported with all fields parsed.
25	delete_rows_event	Fully supported with all fields parsed.
26	incident_event	Unsupported.
27	heartbeat_log_event	Unsupported.
50	table_metadata_event	Specific to Twitter MySQL 5.5.24.t7+. Fully supported with all fields parsed.

Data Types Supported in Row Events

Data Type	Binlog Type	Status
Numeric Types
TINYINT	MYSQL_TYPE_TINY	Fully supported.
SMALLINT	MYSQL_TYPE_SHORT	Fully supported.
MEDIUMINT	MYSQL_TYPE_INT24	Fully supported.
INT	MYSQL_TYPE_LONG	Fully supported.
BIGINT	MYSQL_TYPE_LONGLONG	Fully supported.
FLOAT	MYSQL_TYPE_FLOAT	Fully supported.
DOUBLE	MYSQL_TYPE_DOUBLE	Fully supported.
DECIMAL	MYSQL_TYPE_NEWDECIMAL	Fully supported using BigDecimal.
Temporal Types
TIMESTAMP	MYSQL_TYPE_TIMESTAMP	Fully supported.
DATETIME	MYSQL_TYPE_DATETIME	Fully supported.
DATE	MYSQL_TYPE_DATE	Fully supported.
TIME	MYSQL_TYPE_TIME	Fully supported.
YEAR	MYSQL_TYPE_YEAR	Fully supported.
String Types
CHAR VARCHAR	MYSQL_TYPE_STRING	Fully supported.
TINYBLOB BLOB MEDIUMBLOB LONGBLOB	MYSQL_TYPE_BLOB	Fully supported.
Other Types
ENUM	MYSQL_TYPE_STRING	Supported, but values returned are internal representations.
SET	MYSQL_TYPE_STRING	Supported, but values returned are internal representations.
BIT	MYSQL_TYPE_BIT	Supported, treated as integer of appropriate size.
GEOMETRY	MYSQL_TYPE_GEOMETRY	Supported, treated as BLOB.