Comments (7)
Are you using bulkload importtsv to load data? What is each record size?
bq, put data use 8 reduces,and put 1000 rows of each
You mean 8 reducers in bulkload?
from hindex.
I think bulk load is been used. There is scope for improvement in case of bulk load. We have open issue for that already Rajesh?
from hindex.
Yes Anoop, we have some improvement actions, Once performance testing is done we can commit here that. But not expecting this much degradation. There may be some other problems related to mapreduce?
from hindex.
thanks very very much.
The user table have one cf and 5 column qualifier. it's less then 1kb.
The index table have one index,and in column qualifier in this index.
I just use hadoop mapreduce to put data via hbase's Put in reduce.
when i put data to hbase,that flushing index table‘s memstore to disk is very frequently. Is lock index region and user region?
and i find bulk load package in hindex. So is it better?
from hindex.
@anoopsjohn thanks to answer.
you said bulk load that is bulk load package in hindex, right?
from hindex.
Yes I was thinking that you are using the tool in the bulk load package in HIndex. not?
How is your mappers and reducers?
from hindex.
the mappers get data from the other hadoop cluster, and reducers use hbase native Put to insert data. and i'll try to tool in the bulk load package.
from hindex.
Related Issues (20)
- hindex write issues HOT 7
- Support secure access to the index tables HOT 2
- migrate hindex to hbase 0.96 or higher HOT 5
- deadlock in put operation HOT 3
- issues in put operation HOT 10
- how to use the index? HOT 8
- Error when compile with maven.
- How to use hindex for scanning data? HOT 20
- Steps to implement hindex to my hbase cluster HOT 6
- is there any new release based on hbase0.98.1+? HOT 3
- ERRORs in building with CDH5.1 HOT 10
- Indexing arbitrary column qualifier HOT 4
- Pending items in Arbitrary index feature
- Unify index details validation in preCreateTable and preModifyTable to allow proper indexes
- To determine the split threshold
- Perfomance about hindex in range search
- No compile document? HOT 3
- does hindex only support hbase 0.94.8?
- Your project Huawei-Hadoop hindex is using buggy third-party libraries [WARNING]
- 请求项目如何启动?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hindex.