It doesn't support the format of a DStream like DStream[(String, Map)] about spark-hbase-connector

[nicolaferraro]

Sorry for getting back to you so late...

You can work with any data structure in your RDDs, but you need to convert everything to a plain data structure when reading/writing to hbase.

If your DStream is a (String, Map[String, String]) type, then you can define an implicit converter like:

implicit def myDataWriter: FieldWriter[(String, Map[String, String])] = new FieldWriterProxy[(String, Map[String, String]), (String, Option[String], Option[String], ...)] {

  override def convert(data: (String, Map[String, String])) = (data._1, data._2.get("key1"), data._2.get("key2"), ...) // the first element is the row id

  override def columns = Seq("myid", "key1", "key2", ...)
}

Then the type (String, Map[String, String]) becomes writable and you can write your RDD to HBase inside a dstream.foreachRDD.

The obvious drawback is that you have to enumerate all the columns you want to write data to, so it's not a completely dynamic approach, but this lib prefers compile time checking over dynamic binding.