Comments (6)
OK - i noticed that the key is deleted before it is re-inserted, and therefore there should never be a duplicate.
I'll review my "delete" function - looks like it is buggy.
from sysbench.
The conflict occurs on the primary key. You are right, the INSERT
statement in oltp.lua
immediately follows a DELETE
statement with the same PK value (i
). This way we make sure each transaction is autonomous, i.e. we won't be trying to insert duplicate keys in a single session.
With multiple concurrent sessions nothing guarantees multiple rows with the same IDs cannot be generated by concurrent transactions. In that case, conflicting transactions fail as they should, and sysbench tries to detect such failures and restarts the failed transactions. But exact behavior is implementation-specific, so it's not always detected by sysbench correctly.
MySQL/InnoDB would fail with a deadlock error or a lock wait timeout, and sysbench handles those in the MySQL driver (drv_mysql.c).
PostgreSQL fails with a duplicate key error in such cases due to a different MVCC behavior. See issue #10, especially links to StackExchange discussions for all the gory details. I fixed it by making the PostgreSQL driver handle duplicate key errors by restarting transactions rather than failing the benchmark.
But the problem you are facing brings up a valid issue with the current approach: a non-InnoDB storage engine for MySQL may have a different MVCC behavior. For example, a PostgreSQL-like behavior, which I guess is the case here.
I'm going to reopen the issue and commit an experimental fix for you to test. Thank you.
from sysbench.
My storage engine does not yet properly implement transactions. This leads to a race condition in oltp.lua, when two threads try to insert the same key:
- thread 1 deletes the primary key
- thread 2 deletes the primary key
- thread 2 inserts the primary key
- thread 1 inserts the primary key (fails because it already exists)
A cheap way to fix this would be to make sure that each thread only has its own range of primary keys.
If you come up with a fix then I'll test it. Thanks for all your efforts!
from sysbench.
Oh, I see. In this case you have two opions:
- have sysbench treat your engine as a non-transactional one (like MyISAM) and serialize all concurrent transactions with
LOCK TABLES
:
--- a/sysbench/tests/db/oltp.lua
+++ b/sysbench/tests/db/oltp.lua
@@ -5,7 +5,7 @@ dofile(pathtest .. "common.lua")
function thread_init(thread_id)
set_vars()
- if (((db_driver == "mysql") or (db_driver == "attachsql")) and mysql_table_engine == "myisam") then
+ if (((db_driver == "mysql") or (db_driver == "attachsql")) and (mysql_table_engine == "myisam") or mysql_table_engine == "yourengine") then
begin_query = "LOCK TABLES sbtest WRITE"
commit_query = "UNLOCK TABLES"
else
- or, generate globally unique IDs for the
DELETE
/INSERT
combo:
diff --git a/sysbench/tests/db/oltp.lua b/sysbench/tests/db/oltp.lua
index 985ca0c..9c87091 100644
--- a/sysbench/tests/db/oltp.lua
+++ b/sysbench/tests/db/oltp.lua
@@ -78,7 +78,7 @@ function event(thread_id)
for i=1, oltp_delete_inserts do
- i = sb_rand(1, oltp_table_size)
+ i = sb_rand_uniq(1, oltp_table_size)
rs = db_query("DELETE FROM " .. table_name .. " WHERE id=" .. i)
from sysbench.
The first solution (using LOCK TABLES) works, but i had to change
`begin_query = "LOCK TABLES sbtest WRITE"`
to
`begin_query = "LOCK TABLES sbtest1 WRITE"`
Your second solution (using sb_rand_uniq) also works. I'll use this as a
workaround till I have implemented proper transaction support.
Thanks for your help!
from sysbench.
Thanks for the update. I'll fix the LOCK TABLES
statement in oltp.lua just in case.
from sysbench.
Related Issues (20)
- Sysbench 1.0.20 + scripting disk benchmark HOT 1
- segementation faults when preparing data
- sysbench0.5 test oracle
- Create Sysbench 1.1.0 release tag HOT 1
- [BUG]Segmentation fault(core dumped) error when use --fored-shutdown
- [Question] Would running with `--threads=1` and `--rate=N` cause backpressure, if the database can't keep up with the `rate`?
- sysbench MySQL deadlock?
- `sysbench` command usage documentation not provided
- Only support prepare SelectStmt point query now
- Sysbench CPU is absolutley broken. HOT 1
- build error with mysql 8.3 HOT 1
- `--rand-seed` is non-deterministic
- Getting FATAL: `thread_init' function failed: /usr/share/sysbench/oltp_common.lua:275: SQL API error
- Fail to reconnect to mysql/mariadb when running with mariadb-libs
- How to intercept "Illegal instruction" error.
- Postgres Display actual query error
- Fail to reconnect to mysql when running with more than one thread
- libaio dependency missing from archlinux package
- Bias max_offset in pareto distribution
- Inadequate usage messages
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sysbench.