larsgeorge / hbase-book Goto Github PK

Excuse me, in the HBase1.3 version, how is the row lock explicitly used; the original HTable's lockRow () method and the unlockRow () method seem to have been lost

Hi
How to connect hbase remotely? what config i need to change in hbase server? thanks

		Configuration conf = HBaseConfiguration.create();
		conf.set("hbase.zookeeper.quorum", "192.168.0.174");
		conf.set("hbase.zookeeper.property.clientPort", "2181");
		HBaseAdmin.available(conf);

Above doesn't work
thanks
Peter

How to solve the problem of importing data failure when importing mysql data into hbase (version 2.1.2 or 2.1.3) using sqoop (version 1.4.7)

Could you please add an explicit LICENSE file to the repo so that it's clear under what terms the content is provided, and under what terms user contributions are licensed?

Per GitHub docs on licensing:

Generally speaking, the absence of a license means that the default copyright laws apply. This means that you retain all rights to your source code and that nobody else may reproduce, distribute, or create derivative works from your work. This might not be what you intend.

Thanks!

K[] samples = sampler.getSample(inf, job);
use jdk 1.8
error: incompatible types.
Required: K[]
Found: java.lang.Object[]
auto cast：
K[] samples = (K[]) sampler.getSample(inf, job);

the code in line 93 from ImportFromFile
context.write(new ImmutableBytesWritable(rowkey), put);
has syntax error:
The method write(ImmutableBytesWritable, Writable) in the type TaskInputOutputContext<LongWritable,Text,ImmutableBytesWritable,Writable> is not applicable for the arguments (ImmutableBytesWritable, Put).
by the way, my hadoop-core version is 1.2.1 because I can't get 0.20-append-r1057313 version by maven:

I don't know what's going wrong,can you help me?

Hi Lars,

The compilation of all the source files in the directory is successful after the issual of
the mvn package command.

INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 17 seconds
[INFO] Finished at: Fri Jul 06 17:21:33 CDT 2012
[INFO] Final Memory: 65M/380M
[INFO] ------------------------------------------------------------------------

However when I try to run any of the examples, errors pop up.

varun@varun:~/larsgeorge/hbase-book/ch04$ bin/run.sh client.PutExample
Exception in thread "main" java.lang.NoClassDefFoundError: client/PutExample
Caused by: java.lang.ClassNotFoundException: client.PutExample
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)

What is the issue?Could you please provide your inputs on this ?

I have written a custom Filter extending FilterBase and converted into JAR. The filter looks like this :

public class MyFilter extends FilterBase {
	boolean filterRow= true;
	String srh;
	
	public MyFilter(String str) {
		this.srh= str;
	}

	@Override
	public ReturnCode filterKeyValue(Cell c) throws IOException {
		String str= Bytes.toString(c.getValue());
		
		if(str.contains(str)) {
			filterRow= false;
			return ReturnCode.INCLUDE;
		}
		
		filterRow= true;
		return ReturnCode.SKIP;
	}
	
	@Override
	public  boolean filterRow() {
		return filterRow;
	}

}  

After making the JAR file (i.e. MyFilter.jar) I put it in /use/local/HBase/lib/filters directory. Then I set

export HBASE_CLASSPATH="/usr/local/Hbase/lib/filters/MyFilter.jar"

in hbase-env.sh filter and restarted the hbase server. Then I used the custom filter from java program :

public static void main(String argv[]) throws IOException {
		Configuration conf= HBaseConfiguration.create();
		Connection con= ConnectionFactory.createConnection(conf);
		
		Table table= con.getTable(TableName.valueOf("stud"));
		
		Filter fl= new MyFilter("uc");
		
		Scan sc= new Scan();
		sc.setFilter(fl);
		
		ResultScanner rs= table.getScanner(sc);
		
		for(Result r : rs)
			System.out.println(Bytes.toString(r.getValue(Bytes.toBytes("perData"), Bytes.toBytes("name"))));
	}

But getting following exception

Exception in thread "main" org.apache.hadoop.hbase.DoNotRetryIOException: org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.toFilter(ProtobufUtil.java:1478)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.toScan(ProtobufUtil.java:993)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2396)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33648)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2180)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.toFilter(ProtobufUtil.java:1474)
... 8 more
Caused by: org.apache.hadoop.hbase.exceptions.DeserializationException: parseFrom called on base Filter, but should be called on derived type
at org.apache.hadoop.hbase.filter.Filter.parseFrom(Filter.java:270)
... 13 more

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:329)
at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:408)
at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:204)
at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:65)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:364)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:338)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:136)
at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:65)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.DoNotRetryIOException): org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.toFilter(ProtobufUtil.java:1478)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.toScan(ProtobufUtil.java:993)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2396)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33648)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2180)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.toFilter(ProtobufUtil.java:1474)
... 8 more
Caused by: org.apache.hadoop.hbase.exceptions.DeserializationException: parseFrom called on base Filter, but should be called on derived type
at org.apache.hadoop.hbase.filter.Filter.parseFrom(Filter.java:270)
... 13 more

at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1267)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:34094)
at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:400)
... 10 more

As version 2.0.0,I can not use hbase hbck (version 1) to fix my data inconsistencies, because -fix/ -repair options are NOT supported in version 2.0+ . I can not use hbase hbck2 (version2) to fix ,because it need hbase in bigger than version 2.0.3. It is crazy for me, anyone can solve it.

INFO main - Client environment:java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java
INFO main - Client environment:java.io.tmpdir=/var/folders/g5/08jclb4s7tg4swz4jx31h2sc0000gn/T/
INFO main - Client environment:java.compiler=
INFO main - Client environment:os.name=Mac OS X
INFO main - Client environment:os.arch=x86_64
INFO main - Client environment:os.version=10.9.2
INFO main - Client environment:user.name=yuchao
INFO main - Client environment:user.home=/Users/yuchao
INFO main - Client environment:user.dir=/Users/yuchao/Downloads/hbase-book/hush
INFO main - Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection
INFO main-SendThread() - Opening socket connection to server localhost/fe80:0:0:0:0:0:0:1%1:2181
INFO main-SendThread(localhost:2181) - Socket connection established to localhost/fe80:0:0:0:0:0:0:1%1:2181, initiating session
INFO main-SendThread(localhost:2181) - Session establishment complete on server localhost/fe80:0:0:0:0:0:0:1%1:2181, sessionid = 0x144d2f6553a0008, negotiated timeout = 40000
Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range: -1
at java.lang.String.substring(String.java:1937)
at org.apache.hadoop.hbase.ServerName.parseHostname(ServerName.java:81)
at org.apache.hadoop.hbase.ServerName.(ServerName.java:63)
at org.apache.hadoop.hbase.MasterAddressTracker.getMasterAddress(MasterAddressTracker.java:62)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:568)
at org.apache.hadoop.hbase.client.HBaseAdmin.(HBaseAdmin.java:94)
at com.hbasebook.hush.schema.SchemaManager.process(SchemaManager.java:126)
at com.hbasebook.hush.HushMain.main(HushMain.java:57)

Even after doing mvn package i dont see run.sh coming in bin folder neither it was present in the same when i cloned.

Hi.. I am getting this error when I am trying to parse json...
with follwoing flags
hadoop jar hbasech07.jar ParseJson -i testtable -o testtable
throws this error:

java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:719)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
... 7 more
Caused by: java.lang.NoClassDefFoundError: org/json/simple/parser/JSONParser
at mapreduce.ParseJson$ParseMapper.(ParseJson.java:49)
... 12 more
Caused by: java.lang.ClassNotFoundException: org.json.simple.parser.JSONParser
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
... 13 more