put data is very slowly about hindex

[chrajeshbabu]

Are you using bulkload importtsv to load data? What is each record size?

bq, put data use 8 reduces，and put 1000 rows of each
You mean 8 reducers in bulkload?

[anoopsjohn]

I think bulk load is been used. There is scope for improvement in case of bulk load. We have open issue for that already Rajesh?

[chrajeshbabu]

Yes Anoop, we have some improvement actions, Once performance testing is done we can commit here that. But not expecting this much degradation. There may be some other problems related to mapreduce?

[rexwong]

thanks very very much.

The user table have one cf and 5 column qualifier. it's less then 1kb.
The index table have one index,and in column qualifier in this index.

I just use hadoop mapreduce to put data via hbase's Put in reduce.
when i put data to hbase，that flushing index table‘s memstore to disk is very frequently. Is lock index region and user region?

and i find bulk load package in hindex. So is it better?

[rexwong]

@anoopsjohn thanks to answer.

you said bulk load that is bulk load package in hindex, right?

[anoopsjohn]

Yes I was thinking that you are using the tool in the bulk load package in HIndex. not?
How is your mappers and reducers?

[rexwong]

the mappers get data from the other hadoop cluster, and reducers use hbase native Put to insert data. and i'll try to tool in the bulk load package.