58daojia-dba / mysqlbinlog_flashback Goto Github PK

mysql版本：8.0.18
mysqlbinlog_back版本：0.1.3
有如下测试：

1、新建一个测试表
2、插入2两条数据
3、根据主键更新
mysql> create table t_moshan_test(id int not null auto_increment primary key,age int not null);
Query OK, 0 rows affected (0.01 sec)

mysql> insert into t_moshan_test select 0,1;
Query OK, 1 row affected (0.00 sec)
Records: 1 Duplicates: 0 Warnings: 0

mysql> insert into t_moshan_test select 0,2;
Query OK, 1 row affected (0.00 sec)
Records: 1 Duplicates: 0 Warnings: 0

mysql> update t_moshan_test set age = 1 where id = 2;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0

mysql> update t_moshan_test set age = 10 where id = 2;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0

mysql> select * from t_moshan_test;
+----+-----+
| id | age |
+----+-----+
| 1 | 1 |
| 2 | 10 |
+----+-----+
2 rows in set (0.00 sec)

mysql> update t_moshan_test set age = 11 where id = 2;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0

mysql> select * from t_moshan_test;
+----+-----+
| id | age |
+----+-----+
| 1 | 1 |
| 2 | 11 |
+----+-----+
2 rows in set (0.00 sec)

mysql> update t_moshan_test set age = 12 where id = 2;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0

mysql> select * from t_moshan_test;
+----+-----+
| id | age |
+----+-----+
| 1 | 1 |
| 2 | 12 |
+----+-----+
2 rows in set (0.00 sec)

mysql>

使用工具生成回滚语句如下
/home/moshan/py2env/bin/python mysqlbinlog_back.py --host=172.16.0.20 --username=myuser --port=3306 --password=123456--schema=test --tables=t_moshan_test -S 3306-binlog.000015 -l tmp_sql

#end_log_pos 70122394 2020-04-15T11:02:53 1586919773 3872-binlog.000015;
delete from t_moshan_test where id=1;
#end_log_pos 70122743 2020-04-15T11:02:54 1586919774 3872-binlog.000015;
delete from t_moshan_test where id=2;
#end_log_pos 70123863 2020-04-15T11:05:10 1586919910 3872-binlog.000015;
update t_moshan_test setage=2,id=2 where id=1;
#end_log_pos 70124241 2020-04-15T11:05:15 1586919915 3872-binlog.000015;
update t_moshan_test setage=2,id=1 where id=10;
#end_log_pos 70126105 2020-04-15T11:08:52 1586920132 3872-binlog.000015;
update t_moshan_test setage=2,id=10 where id=11;
#end_log_pos 70126483 2020-04-15T11:08:56 1586920136 3872-binlog.000015;
update t_moshan_test setage=2,id=11 where id=12;

看这个回滚语句明显是有问题的，而且能稳定复现，有空请 帮忙看看

具体情况如下：
DB version:MySQL 5.6.29 64bit 社区版
OS: RHEL 6.4 x86_64

1、解析出来的SQL
[root@rac3 log]# cat save_data_dml_test_20161206_164530.sql
#end_log_pos 1014 2016-12-06T16:42:36 1481013756 mysql-bin.000001;
insert into _wuyong_keep_data_(bfr_sex,bfr_id,bfr_user_id,op_datetime,op) values(None,1,2,'2016-12-06 16:42:36','delete');

2、说明
--sex字段如果有空值，对应的空值会被解析成None

3、执行插入会报如下错误，导致插入失败
mysql> insert into _wuyong_keep_data_(bfr_sex,bfr_id,bfr_user_id,op_datetime,op) values(None,1,2,'2016-12-06 16:42:36','delete');
ERROR 1054 (42S22): Unknown column 'None' in 'field list'

4、建议，将空值解析成null，即可避免这个问题的出现。
mysql>insert into _wuyong_keep_data_(bfr_sex,bfr_id,bfr_user_id,op_datetime,op) values(null,1,2,'2016-12-06 16:42:36','delete');
Query OK, 1 row affected (0.11 sec)

===log will also write to .//mysqlbinlog_flashback.log===
parameter={'start_binlog_file': 'mysql-bin.043654', 'stream': None, 'keep_data': True, 'file': {'data_create': None, 'flashback': None, 'data': None}, 'add_schema_name': False, 'start_time': None, 'keep_current_data': False, 'start_to_timestamp': 1680229920, 'mysql_setting': {'passwd': 'weflow@pms', 'host': '9.148.120.87', 'charset': 'utf8', 'port': 3306, 'user': 'root'}, 'table_name': 'ACT_GE_BYTEARRAY', 'skip_delete': False, 'schema': 'test_full_link_1', 'stat': {'flash_sql': {}}, 'table_name_array': ['ACT_GE_BYTEARRAY'], 'one_binlog_file': False, 'output_file_path': './log', 'start_position': 4, 'skip_update': False, 'dump_event': False, 'end_to_timestamp': 1680230400, 'skip_insert': False, 'schema_array': ['test_full_link_1']}
scan 10000 events ....from binlogfile=mysql-bin.043654,timestamp=2023-03-31T10:32:22
scan 20000 events ....from binlogfile=mysql-bin.043654,timestamp=2023-03-31T10:32:24
scan 30000 events ....from binlogfile=mysql-bin.043654,timestamp=2023-03-31T10:32:24
scan 40000 events ....from binlogfile=mysql-bin.043654,timestamp=2023-03-31T10:32:30
error:'ascii' codec can't decode byte 0xe5 in position 673: ordinal not in range(128)
=====Additional info:dump stack to diagnose =======
Traceback (most recent call last):
File "mysqlbinlog_back.py", line 190, in main
deal_all_event(parameter)
File "/mysqlbinlog_flashback/flashback.py", line 201, in deal_all_event
deal_delete_rows(event,parameter.file,parameter.stat,logfile=stream.log_file,add_schema_name=parameter.add_schema_name)
File "/mysqlbinlog_flashback/flashback.py", line 306, in deal_delete_rows
sql=joint_insert_sql(event.schema,event.table,event.primary_key,row,add_schema_name)
File "/mysqlbinlog_flashback/joint_sql.py", line 41, in joint_insert_sql
(columns,values)=generate_two_array_column_and_value(row["values"])
File "/mysqlbinlog_flashback/joint_sql.py", line 231, in generate_two_array_column_and_value
logger.debug(u"dump row ele: {0}={1},type={2}".format(key,row[key],type(row[key])))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 673: ordinal not in range(128)