Insert data with spark, some data can not be queried out about hgraphdb

[rayokota]

Do you get the same result if you set .setBulkLoaderSkipWAL(false)?

[youzp]

@rayokota thanks, today i will try it.

[youzp]

@rayokota Sorry for the delay, i have tried several times after set .setBulkLoaderSkipWAL(false), when the amount of data reaches more than one billion, data missing from each query will appear. Are there other ways to ensure that data is not lost?

[rayokota]

If you are using vertex indices, please try again with 2.0

[youzp]

Maybe I have found the reason for the missing data.An abnormal data containing '\u0000' causes asynchronous process of a piece of data to fail.

ERROR AsyncProcess: Cannot get replica 0 location for

java.lang.IllegalArgumentException: Cannot encode String values containing '\u0000'
at org.apache.hadoop.hbase.util.OrderedBytes.encodeString(OrderedBytes.java:963)
at io.hgraphdb.ValueUtils.serialize(ValueUtils.java:196)
at io.hgraphdb.models.EdgeIndexModel.serializeForWrite(EdgeIndexModel.java:363)
at io.hgraphdb.mutators.EdgeIndexWriter.constructPut(EdgeIndexWriter.java:58)
at io.hgraphdb.mutators.EdgeIndexWriter.lambda$constructInsertions$55(EdgeIndexWriter.java:51)
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.HashMap$EntrySpliterator.tryAdvance(HashMap.java:1712)
at java.util.stream.StreamSpliterators$WrappingSpliterator.                                              lambda$initPartialTraversalState$0(StreamSpliterators.java:294)
at java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:          206) 
at java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:161)
at java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:300)
at java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
at org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils$2.hasNext(IteratorUtils.java:209)
at org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils.fill(IteratorUtils.java:62)
at org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils.list(IteratorUtils.java:85)
at io.hgraphdb.HBaseBulkLoader.getMutationList(HBaseBulkLoader.java:219)
at io.hgraphdb.HBaseBulkLoader.indexEdge(HBaseBulkLoader.java:142)
at io.hgraphdb.HBaseBulkLoader.addEdge(HBaseBulkLoader.java:125)

[rayokota]

Thanks for letting me know. Closing this issue.