scan.setAttribute() question about hindex

[chrajeshbabu]

Can u give the information of indices you have created and indices u have used in the expression?

[rexwong]

thanks for reply!
i copy create indices code from mapreduce in hindex.So this method is like this:

    public static IndexedHTableDescriptor createIndexedHTableDescriptor(String tableNameToIndex,
                                                                        String input, String[] cfs) {
        IndexedHTableDescriptor indexHTableDesc = new IndexedHTableDescriptor(tableNameToIndex);
        List<String> colFamilyList = new ArrayList<String>();
        for (String cf : cfs) {
            HColumnDescriptor hColumnDescriptor = new HColumnDescriptor(cf);
            indexHTableDesc.addFamily(hColumnDescriptor);
            colFamilyList.add(hColumnDescriptor.getNameAsString());

        }
        if (input != null) {
            String[] indexSplits = input.split("#");
            for (String index : indexSplits) {
                String[] indexName = index.split("=>");
                if (indexName.length < 2) {
                    System.out.println("Invalid entry.");
                    System.exit(-1);
                }
                IndexSpecification iSpec = new IndexSpecification(indexName[0]);

                String[] cfSplits = indexName[1].split(";");
                if (cfSplits.length < 1) {
                    System.exit(-1);
                }
                else {
                    for (String cf : cfSplits) {
                        String[] qualSplits = cf.split(":");
                        if (qualSplits.length < 2) {
                            System.out.println("The qualifiers are not given");
                            System.exit(-1);
                        }
                        if (!colFamilyList.contains(qualSplits[0])) {
                            System.out.println("Valid CF not found");
                            System.exit(-1);
                        }
                        String[] qualDetails = qualSplits[1].split(",");
                        for (String details : qualDetails) {
                            String substring = details.substring(1, details.lastIndexOf("]"));
                            if (substring != null) {
                                String[] splitQual = substring.split("->");
                                if (splitQual.length < 2) {
                                    System.out.println("Default value length and data type will be take");
                                    iSpec.addIndexColumn(new HColumnDescriptor(qualSplits[0]), splitQual[0],
                                            ValueType.String, Constants.DEF_MAX_INDEX_NAME_LENGTH);
                                } else {
                                    String[] valueType = splitQual[1].split("&");
                                    iSpec.addIndexColumn(new HColumnDescriptor(qualSplits[0]), splitQual[0],
                                            ValueType.valueOf(valueType[0]), Integer.parseInt(valueType[1]));
                                }
                            }
                        }
                    }
                }
                indexHTableDesc.addIndex(iSpec);
            }
        }
        return indexHTableDesc;
    }

i use below code to inovke

String input = "IDX1=>cf1:[a2->String&15],[a3->String&20],[a9->String&2]";
String[] cf = new String[]{"cf1"};
IndexedHTableDescriptor idxHtable = TestUtils.createIndexedHTableDescriptor("index_test_2",input,cf);
HBaseAdmin hBaseAdmin = new HBaseAdmin(configuration);
hBaseAdmin.createTable(idxHtable,RegionSplitsUtil.splits("000","fff",20));

right now,i have more important thing to ask, it's about dynamically add or drop index. do you have any plan about this feature?

[anoopsjohn]

You are creating and index IDX1 on 3 columns (a2, a3, a9)
In the query u have an equal condition on a2 and range condition on a9. When you explicitly say which index to be used (using the scan.setAttribute(Constants.INDEX_EXPRESSION,..)) there is no index evaluation happening in server side and blindly believing the user passed info. Actually in your query case the index can not be used as the query is missing the a3 column.
For a2 value (equal condition) alone, the index can be used. So on all rows selected (using index) with given value of a2, the a9 conditions are applied and further filtering will happen.. This will happen when you are not explicitly passing the index details in Scan using setAttribute. The HInex server side code having the intelligence to decide these.

Why we allow the user to pass the index details in setAttribute I can tell with an example.
Consider there are many indices on a table and a query comes with conditions on many columns. We decide which all indices to be used for this scan at server side. What we give importance there is to have min no# indices to be used with max coverage on column conditions. As we dont have any stat info abt these index tables, this can be bit wrong in some cases. So an advanced user might be knowing what is optimum combination of indices to be used. This setAttribute is for those users. In case of other DBs also a user, if he wish, can add HINT to use which all indexes to be used while scanning. If nothing is specified as this, then the logic in server will decided what is the optimum combination which can be used. Am I explaining things clearly for you?

So as such this issue is not a valid one and I will close

[anoopsjohn]

Closing as INVALID as per the explanation given above.