issues writing to HBase version 0.94.2-cdh4.2.0 about spark-hbase-connector

[xiaolezhu]

I tried using two ways to write to hbase:

1. standard spark API saveAsNewAPIHadoopDataset.
2. use spark-hbase-connector.

They both failed with the same issue that it was hanging there after "Created table instance for testTable" without any error messages:

Log
16/03/24 23:26:00 INFO ClientCnxn: Session establishment complete on server bpaascp1devhbasezk01.bpaas.local/192.168.23.33:2181, sessionid = 0x152844be97f6ddf, negotiated timeout = 40000
16/03/24 23:26:00 INFO ZooKeeperRegistry: ClusterId read in ZooKeeper is null
16/03/24 23:26:00 INFO TableOutputFormat: Created table instance for testTable

By looking into the log closely - there is one more line of Log says: ClusterId read in ZooKeeper is null. The issue is because that *zookeeper.znode.parent * was not set with proper value - *zookeeper.znode.parent * tells which znode keeps the data (and address for HMaster) for the cluster.

The value of zookeeper.znode.parent in HBASE_CONF/hbase-site.xml. After setting the proper value into HBaseConfiguration, the issue was gone by using saveAsNewAPIHadoopDataset.

However, in spark-hbase-connector there will need to be code changes to set value of zookeeper.znode.parent, similar to how spark.hbase.host is set.

[avihoo]

I'm afraid I don't have the zookeeper.znode.parent property set in my HBASE_CONF/hbase-site.xml.
What should I do in order to make it still save from spark to hbase?

[nicolaferraro]

Version 1.0.3 of the connector (already available on maven) supports the hbase-site.xml file for specifying a custom configuration.

[xiaolezhu]

Great to know :)