mkjellman / perlcassa Goto Github PK

Hello,
I am using perlcassa 0.060. The cassandra DB I am interfacing with has several columns of the timestamp type: scheduled_time, generation_start_time, and generation_end_time. When I execute a select query that includes one of those columns, I get this error behavior:

EV: error in callback (ignoring): [ERROR] Attempted to unpack unimplemented data type. (timestamp) at /opt/3rdParty/lib/perl5/vendor_perl/5.16.3/perlcassa/Decoder.pm line 171.
at /opt/3rdParty/lib/perl5/vendor_perl/5.16.3/x86_64-linux-thread-multi/EV.pm line 1205.
EV::ANON/opt/3rdParty/lib/perl5/vendor_perl/5.16.3/x86_64-linux-thread-multi/EV.pm:1206 called at /opt/3rdParty/lib/perl5/vendor_perl/5.16.3/x86_64-linux-thread-multi/AnyEvent/Impl/EV.pm line 88
eval {...} called at /opt/3rdParty/lib/perl5/vendor_perl/5.16.3/x86_64-linux-thread-multi/AnyEvent/Impl/EV.pm line 88
AnyEvent::CondVar::Base::_wait('AnyEvent::CondVar=HASH(0x45d6910)') called at /opt/3rdParty/lib/perl5/vendor_perl/5.16.3/x86_64-linux-thread-multi/AnyEvent.pm line 1981
AnyEvent::CondVar::Base::recv('AnyEvent::CondVar=HASH(0x45d6910)') called at /opt/3rdParty/lib/perl5/vendor_perl/5.16.3/perlcassa/Binary/Socket.pm line 277

I would like to know if this is "on the radar" for being fixed in a future version, and if there is a workaround I can use in the meantime to access those columns of the timestamp data type.
Thank you, and best regards,
gesterob

Hi

execute_cql3_query methode is mising from perlcassa.pm. Please share its implemetation. Just browsing i fount its 0.5 version which include above method in perlcassa.pm

But now running the testcases is throwing
TApplicationException=HASH(0x9aa82c0)# Looks like your test exited with 255 just after 1. error

Running

$res = $obj->exec("CREATE KEYSPACE $test_keyspace WITH REPLICATION = {'CLASS' : 'SimpleStrategy', 'replication_factor': 1};");
ok($res, "Create test keyspace ($test_keyspace).");
Throwing TApplicationException=HASH(0x9aa82c0)# Looks like your test exited with 255 just after 1. error

Please help me with the solution

Thanks
Geetanjali

Is perlcassa compatible with datastax cassandra 3.0 ?.

while trying to use below query with perlcassa
my $res = $obj->exec("SELECT * FROM $test_keyspace.$cfname1");
throws following error:

Full error message: $VAR1 = bless( {
'code' => 1,
'message' => 'Invalid method name: 'execute_cql3_query''
}, 'TApplicationException' );

Please provide the solution for this

Thanks,
Geetanjali

Hey Kjellman I will make fix this and make some test cases to prevent regression.

Essentially perl treats things as utf8 when it should not. This breaks various decoding routines which assume byte strings and not character strings.

One example is a DataType with value "00000140114cdd9e" looks like "00000140114c\x{dd9e}" or "@lݞ". Any type that has a value that might look like a utf8 encoded string could be affected.

@rg1 brought up the issue that the pack_UTF8 function does not properly pack UTF8. The function fails to convert the UTF8 characters into bytes. This causes problems when UTF8 strings are packed into a set or list collection type. Since perl's length or pack("n/") normally count characters instead of bytes.

The fix is short and understandable, but I am creating this bug in order to track adding test cases.

[WARNING] attempt 0 of 2 - unable to connect to host 10.8.30.101 at /brts/lib/perlcassa/Client.pm line 237.
[Tue Jan 29 18:22:40 2013] [notice] child pid 28925 exit signal Segmentation fault (11)

CQL3 exec(): parameter hash doesn't work for int, uuid and timeuuid type columns as these types don't accept to be quoted. Possibly affects other types as well.
Currenty these types must be inlined into the query.

Example of code showing the issue:

#!/usr/bin/perl


use perlcassa;
use UUID::Tiny;
use Try::Tiny;


my $cassy = new perlcassa(
    'keyspace' => "testspace",
    'seed_nodes' => ['localhost'],
    'write_consistency_level' => Cassandra::ConsistencyLevel::QUORUM,
    'read_consistency_level' => Cassandra::ConsistencyLevel::QUORUM,
    'port' => '9160'
);

try {
    $cassy->exec(
        "CREATE TABLE testtable (key uuid," .
        " ivalue int," .
        " tvalue text," .
        " PRIMARY KEY(key))"
    );
} catch {
    print "Probably table existed before\n";
};


my $key = create_UUID_as_string();
try {
    $cassy->exec(
        "INSERT INTO testtable (key, ivalue, tvalue)" .
        " VALUES ($key, 123, 'teststring')"
    );
    print "Test 1 succeeded\n";
} catch {
    print "Test 1 failed: $_\n";
};

$key = create_UUID_as_string();
try {
    $cassy->exec(
        "INSERT INTO testtable (key, ivalue, tvalue)" .
        " VALUES (:keyvalue, 234, 'teststring')",
        {
            keyvalue => $key
        }
    );
    print "Test 2 succeeded\n";
} catch {
    print "Test 2 failed: $_\n";
};

$key = create_UUID_as_string();
try {
    $cassy->exec(
        "INSERT INTO testtable (key, ivalue, tvalue)" .
        " VALUES ($key, :ivalue, 'teststring')",
        {
            ivalue => 345,
        }
    );
    print "Test 3 succeeded\n";
} catch {
    print "Test 3 failed: $_\n";
};

$key = create_UUID_as_string();
try {
    $cassy->exec(
        "INSERT INTO testtable (key, ivalue, tvalue)" .
        " VALUES ($key, 456, :tvalue)",
        {
            tvalue => 'teststring2'
        }
    );
    print "Test 4 succeeded\n";
} catch {
    print "Test 4 failed: $_\n";
};

EDIT: for perl highlighting

I am getting the following error from time to time, which however cannot reproduce. It just happens and when I run my script after sometime it succeeds. The data is inserted to my cluster but this error is thrown anyway. Does anybody else have come across this issue? Using perlcassa v0.050. Thank you

Please brief what all functionality can be used with cql3 calls
in terms of
data type support,
types of query ,
functions
etc . which can and cannot be invoked using cql3

Thanks
Geetanjali

I'm looking for a way to bulk insert into one column family with different keys and different col-keys:
e.g.:
column family "test"
"key1" => "col1" => "data1"
"key2" => "col1" => "data2"
"key3" => "col2" => "data3"
...

bulk_insert() is missing the functionality to set a different key on every mutation, as far as I can see.

I would implement that myself, but which way?
a) add a new method, something like "bulk_insert2()"?
b) change bulk_insert(), making the new version incompatible to the old one?
c) write some kind of glue code inside bulk_insert() to switch behavior (e.g. based on "exists($opts{key})"?
d) any other ideas?!

Any ideas or hints for me?

Setup:
Cassandra cluster with two nodes. Trying to execute a select command to the cluster, when one node is down (keyspace made with replication factor 2), by calling $perlcassa->exec($select_query). It throws 'Cassandra::UnavailableException'

Description:
It seems that the 'Cassandra::Compression::NONE' option, when calling $client->execute_prepared_cql3_query (lines 378 and 386) is not used. As a result the 'Cassandra::ConsistencyLevel::ONE' is not parsed properly. I would expect my select query to be executed when one node is down.

Solving the problem:
Just commenting the lines 378 and 386 in perlcassa.pm . Btw, should 'Cassandra::ConsistencyLevel::ONE' be hard-coded in the function?

Used:
Cassandra 1.2
ActivePerl 5.14.2
perlcassa 0.50

Thank you,
George

heya,
just a heads up that the use of threads in perl is pretty much avoided at all costs.
Maybe take a look at IO::Async ( event loop based )? Or if you want, I'd totally lend a hand.

Is there any way to retrieve rows with a CompositeType as key on perlcassa? I have not seen that use case on the examples yet.

I'm using Cassandra 1.2.5. Using masters from github for perlcassa and thrift-xs
When I try to access a column family(now called table) that I created via cqlsh I get the mesasge: Unable to find the column family users in ks1 at /usr/local/share/perl/5.14.2/perlcassa.pm line 510.

However if I create the column family via cassandra-cli, then perlcassa works without problems.

Description:
Cassandra cluster with two nodes. ONE OF THEM IS DOWN. Using perlcassa within a thread. Just the presence of 'use DBI' in the code causes a segmentation fault. Looks like the problem is in the unless block within the "fail_host" function in the Client.pm. Using ActiveState perl 5.14 and perlcassa 0.50

Sample code (if you comment 'use DBI' the code will work):

use strict;
use warnings;

use threads;
use perlcassa;
use DBI;

threads->create('start_thread')->join();

sub start_thread {
    my $driver = new perlcassa(
        keyspace => 'mykeyspace',
        hosts => ['host1','host2'],
        do_not_discover_peers => 1,
   );

    $driver->exec("Select id from mytable limit 10");
}

Values are not currently being packed, at all.

Check perlcassa.pm lines 579 and 602.

If there is only one node in the cluster the seed_nodes option does not work.

This may be related to the fact that the system.peers table does not contain the node that runs the query.

my $obj = new perlcassa(
    'columnfamily' => 'myCF',
    'keyspace' => 'myKeyspace',
    'seed_nodes' => ['host1.cassandra.local'],
);

Is it possible to insert a null value using a client exec function?

Example:

$obj->exec("INSERT INTO cf1(key, value) VALUES(:key, :value)", {key => 1, value => NULL?? });

thx

The perlcassa library currently lacks the ability to manipulate keyspaces.

I propose two new methods, create_keyspace and drop_keyspace.

create_keyspace(
  name => 'ks_name',
  # Optional
  replication_strategy => 'SimpleStrategy',
  durable_writes => 1,
  strategy_options => undef,
);

drop_keyspace(
  name => 'ks_name'
);

According to the Cassandra DataModel the convention for timestamps is to use microseconds.

Because perlcassa uses seconds, a deletion or insertion from the cassandra cli or cqlsh cannot be overwritten by perlcassa. Switching to microseconds will make perlcassa compatible with the cli and cqlsh, which both use microseconds.

When using CQL and doing a SELECT on an UUIDType column, it returns a short int containing only the first 2 bytes of the UUID.

Below a solution proposal returning a formated UUID string. Adding the CQL3 TimeUUIDType by the way. An other solution would be to return some sort of 128 bit binary structure.

diff --git a/lib/perlcassa/Decoder.pm b/lib/perlcassa/Decoder.pm
index aee25ef..fe56cf2 100644
--- a/lib/perlcassa/Decoder.pm
+++ b/lib/perlcassa/Decoder.pm
@@ -26,7 +26,6 @@ our %simple_unpack = (
        'Int32Type'             => 'l>',
        'LongType'              => 'q>',
        'UTF8Type'              => 'a*',
-       'UUIDType'              => 'S',
        'CounterColumnType'     => 'q>'
 );

@@ -34,6 +33,8 @@ our %complicated_unpack = (
        'IntegerType'           => \&unpack_IntegerType,
        'DecimalType'           => \&unpack_DecimalType,
        'InetAddressType'       => \&unpack_ipaddress,
+       'UUIDType'                      => \&unpack_uuid,
+       'TimeUUIDType'          => \&unpack_uuid
 );

 sub new {
@@ -198,6 +199,21 @@ sub unpack_ipaddress {
     return $ret;
 }

+# Unpack uuid/uuidtime type
+# Returns a string
+sub unpack_uuid {
+    my $packed_value = shift;
+    my $data_type = shift;
+    my $len = length($packed_value);
+    my $ret;
+    if ($len ==16) {
+               $ret = unpack('H8', $packed_value).'-'.unpack('x4H4', $packed_value).'-'.unpack('x6H4', $packed_value).'-'.unpack('x8H4', $packed_value).'-'.unpack('x10H12', $packed_value);
+       } else {
+        die("[ERROR] Invalid uuid type.");
+    }
+    return $ret;
+}
+
 # Unpack a collection type. List, Map, or Set
 # Returns:
 #   array - for list

Hi Micheal ,

I am trying to use Perl module -Perlcassa

My Use Case is :
Fetch a query from Cassandra using perl through get and get_slice where rowkey is defined as longtype.

Issue :
Row_key value described in table is long type whereas cassandra.pm module is sending it as a string.

Error on running script:
$VAR1 = bless( {
'why' => 'Expected 8 or 0 byte long (11)'
}, 'Cassandra::InvalidRequestException' );

Column family description of Cassandra:

create column family Users
with column_type = 'Standard'
and comparator = 'UTF8Type'
and default_validation_class = 'BytesType'
and key_validation_class = 'LongType'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy = 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and column_metadata = [
{column_name : 'st',
validation_class : BytesType},
{column_name : 'pwrCalUsr',
validation_class : BooleanType},
{column_name : 'photoURL',
validation_class : UTF8Type},
{column_name : 'uNm',
validation_class : UTF8Type},
{column_name : 'netIDs',
validation_class : UTF8Type},
{column_name : 'pronoun',
validation_class : UTF8Type},
{column_name : 'wrkRmNo',
validation_class : UTF8Type},
{column_name : 'handle',
validation_class : UTF8Type},
{column_name : 'mNm',
validation_class : UTF8Type},
{column_name : 'clntPw',
validation_class : UTF8Type},
{column_name : 'lNm',
validation_class : UTF8Type},
{column_name : 'availId',
validation_class : IntegerType},
{column_name : 'atMobile',
validation_class : BooleanType},
{column_name : 'tmZn',
validation_class : UTF8Type},
{column_name : 'hmAddr',
validation_class : UTF8Type},
{column_name : 'inMeeting',
validation_class : BooleanType},
{column_name : 'wrkAddr',
validation_class : UTF8Type},
{column_name : 'fNm',
validation_class : UTF8Type},
{column_name : 'onThePhone',
validation_class : BooleanType},
{column_name : 'dept',
validation_class : UTF8Type},
{column_name : 'lstModTm',
validation_class : DateType},
{column_name : 'presence',
validation_class : UTF8Type},
{column_name : 'uId',
validation_class : LongType},
{column_name : 'title',
validation_class : UTF8Type},
{column_name : 'supervisorPersonID',
validation_class : IntegerType},
{column_name : 'co',
validation_class : UTF8Type},
{column_name : 'available',
validation_class : BooleanType},
{column_name : 'offline',
validation_class : BooleanType},
{column_name : 'crTm',
validation_class : DateType},
{column_name : 'availty',
validation_class : IntegerType},
{column_name : 'presenceId',
validation_class : IntegerType},
{column_name : 'winTmZn',
validation_class : UTF8Type},
{column_name : 'empId',
validation_class : UTF8Type},
{column_name : 'photoID',
validation_class : IntegerType}]
and compression_options = {'sstable_compression' : 'org.apache.cassandra.io.compress.SnappyCompressor'};

Thanks,
Swati

i have table like

CREATE TABLE test_all(key text PRIMARY KEY, country text, state text, city text,month_of_year int, store_name text, in_stock boolean, store_city text, store_state float,year int);

Want to bulk insert the data using bulk_insert of perlcassa
my %bulk = (
#'key' => ['Tri-State Green Pepper'],
'country' => ['USA'],
'state' => ['WA'] ,
'city' => ['Seattle'],
'month_of_year' => ['8] ,
'store_name' => ['Store 99'] ,
'in_stock' => ['0'],
'store_city' => ['WA'] ,
'store_state' => ['Novato'] ,
'year' => ['2020']
);

$obj->bulk_insert(
'key' => 'Tri-State Green Pepper',
'columns' => %bulk
);

With this getting below error
$VAR1 = '[8] does not fit inside a 32-bit int at /usr/local/share/perl/5.10.1/perlcassa.pm line 1341.
$VAR1 = '[2020] does not fit inside a 32-bit int at /usr/local/share/perl/5.10.1/perlcassa.pm line 1341.

If i comment month_of_year and year value then it shows no error but data is not inserted in the table.

Key for my new row will be ''Tri-State Green Pepper', do i need to mention in my hash

Please share the code to work this scenario.

Also share the code for the same to insert the data as shown in "compositetype_get_example.pl "

Thanks,
Geetanjali

Most of the low level support for prepared queries is ready.

Things that are still missing:

• Syntax
• Map type support
• List type support
• Set typesupport

Does perlcassa support authentication like the SimpleAuthenticator? I did not see a way to specify user credentials.

There seems to be no way to access counter column data or to unpack more than one type of columnfamily data.

Calls to _unpack_value are always given the column rather than counter_column data. The calls are also not passed the column family option.

Hey Kjellman,

Do you think you could add me on CPAN as a co-maintainer of perlcassa? My account on CPAN is cblower.

timestamp should not be undef

I am having a problem with perlcassa -
My Cassandra daemon is up -

I installed Cassandra (1.2.6.6) via the community version of DSE 3.1;
I launched the Cassandra service, and see the process daemon running. The CLI is working fine.
I installed also perlcassa on another machine as I want to experiment with it. (connection to cassandra daemon with port 9160 is working)
After installing Perlcassa, I tried to run the example (changing seed_nodes attributes to fit my needs) , but it says:

perl simple_insert.pl
[ERROR] There were no available Cassandra servers left at /usr/local/share/perl5/perlcassa/Client.pm line 123.

Can anybody help?

I wanted to create a table as shown below:
CREATE TABLE test_all(key text PRIMARY KEY, country text, state text, city text, store_name text, in_stock boolean, store_city text, store_state float,year int)

How shall i pass the column names and their types to the below method

$obj->column_family(
'action' =>'create', # or 'update' or 'drop'
'columnname' => 'cfname',
'keyspace' => 'myKeyspace', #optional and not needed it specified in object creation
'comparator_type' => 'CompositeType(UTF8Type, UTF8Type)',
'key_validation_class => 'UTF8Type',
'default_validation_class => 'UTF8Type'
);

Please share the exact syntax to create the table as mentioned above.

Thanks,
Geetanjali

Silly little issue. When connecting to a cassandra cluster with only one node seed_nodes currently does not work.

[ERROR] There were no available Cassandra servers left at .../perlcassa/lib/perlcassa/Client.pm line 119.

It looks like we do not add the seed_node used to the list of available hosts.

Hey, I notice the approach you are taking with your API is very close to some of the aspects https://github.com/zznate/intravert-ug is offering. Essentially working with simple objects like lists, hashes, etc. Doing type encoding etc. Just wanted to say good work, and that maybe what we are doing fits together very well.

My Cassandra DB uses internal authentication, so I need to pass in a
username/password in order to connect? I don't see a way to do this
in percassa.