demerphq / data-consumer Goto Github PK

Synopsis has this code snippet:

$consumer->consume( sub {
    my $id = shift;
    print "processed $id\n";
} );

This does not match description of consume() later in the pod, where it says consume callback receives ($consumer, $id, ...).

Default query for MySQL uses get_lock function in the WHERE clause to find rows that are not locked by any other process and to lock one of them at the same time. E.g.:

select ... where flag=0 and get_lock(concat_ws('=', 'prefix', id))!=0

It relies on the fact that a database session is only allowed to hold one lock at a time, and get_lock is automatically releasing any previously held locks. This behaviour ensures that after this query is executed, worker has a row that isn't locked by anyone else and it has only one row locked.

This no longer works correctly under MySQL 5.7, which removed one-lock-per-session limitation. Now worker can end up locking as much as all rows in a table, preventing all other workers from claiming any items in the queue.

i.e. if you instantiate Data::Consumer::MySQL without specifying state values, like so:

my $consumer = Data::Consumer::MySQL->new(  
  dbh => $my_awesome_dbh,
  table => 'MyAwesomeTable',
); 

...then the item's flag_field will change to 1 as it's acquired by a consumer, and then remain at that value forever.

The docs say if you don't specify values for the different states, then the defaults are 0 .. 3, but this apparently isn't true :)