Comments (13)
Looks like some issue with communication from the CLI to the WIndowingServer. So from the CLI a process is spawned to run the WindowingEngine. The first thing this does is send the CLI the port to commute on; this initial communication is failing.
Can you post the windowingCli.sh you are using.
from sqlwindowing.
THISSERVICE=windowingCli
export SERVICE_LIST="${SERVICE_LIST}${THISSERVICE} "
windowingCli () {
CLASS=com.sap.hadoop.windowing.WindowingHiveCliDriver
JAR=/tmp/windowing/com.sap.hadoop.windowing-.jar
#JAR=${HIVE_LIB}/com.sap.hadoop.windowing-.jar
exec $HADOOP jar $JAR
}
windowingCli_help () {
windowingCli "--help"
}
from sqlwindowing.
Ameet, are you also having this issue?
from sqlwindowing.
sorry, didn't see the thread started by someone else. No, it works fine if I use metaservice. but not without it
from sqlwindowing.
The .sh file is the same as mine. How and what is metaservice? I didn't use that.
Yong
Date: Sun, 12 Feb 2012 18:28:59 -0800
From: [email protected]
To: [email protected]
Subject: Re: [SQLWindowing] can not start the windowCli (#11)sorry, didn't see the thread started by someone else. No, it works fine if I use metaservice. but not without it
Reply to this email directly or view it on GitHub:
#11 (comment)
from sqlwindowing.
He is talking about the hive metastore server.
But your stacktrace indicates that you didn't get that far.
Can you check the following things
- Your .sh points to com.sap.hadoop.windowing-0.0.1-SNAPSHOT.jar
- your -w option points to com.sap.hadoop.windowing-0.0.1-SNAPSHOT-jar-with-dependencies.jar
from sqlwindowing.
you need a metaservice to use this jar,
- on another machine, start the metaservice, : hive --service metastore
- on the first machine, change the hive-site.xml to reflect a connection to that metaservice,
add these 2 sections,
hive.metastore.local
false
controls whether to connect to remove metastore server or open a new metastore server in Hive Client JVM
from sqlwindowing.
It is fresh installed hive + hadoop. The only thing I changed is to use the mysql as the metastore in the hive-site.xml
~/hive/bin/ext> more windowCli.sh # Licensed to the Apache Software Foundation (ASF) under one or more# contributor license agreements. See the NOTICE file distributed with# this work for additional information regarding copyright ownership.# The ASF licenses this file to You under the Apache License, Version 2.0# (the "License"); you may not use this file except in compliance with# the License. You may obtain a copy of the License at## http://www.apache.org/licenses/LICENSE-2.0## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.# See the License for the specific language governing permissions and# limitations under the License.
THISSERVICE=windowingCliexport SERVICE_LIST="${SERVICE_LIST}${THISSERVICE} "
windowingCli () { CLASS=com.sap.hadoop.windowing.WindowingHiveCliDriver JAR=${HIVE_LIB}/com.sap.hadoop.windowing-0.0.1-SNAPSHOT-jar-with-dependencies.jar exec $HADOOP jar $JAR
windowingCli_help() { windowingCli "--help"}
~/hive/bin> ./hive --service windowingCli -w com.sap.hadoop.windowing-0.0.1-SNAPSHOT-jar-with-dependencies.jarHive history file=/tmp/yzhang/hive_job_log_yzhang_201202141330_1899362727.txtException in thread "main" com.sap.hadoop.windowing.WindowingException: java.lang.NumberFormatException: null at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:77) at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:102) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:54) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:182) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:190) at com.sap.hadoop.windowing.cli.WindowingClient.connect(WindowingClient.groovy:126) at com.sap.hadoop.windowing.cli.WindowingClient.(WindowingClient.groovy:38) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:77) at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:102) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:54) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:182) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:198) at com.sap.hadoop.windowing.WindowingHiveCliDriver.setupClient(WindowingHiveCliDriver.groovy:64) at com.sap.hadoop.windowing.WindowingHiveCliDriver$setupClient.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120) at com.sap.hadoop.windowing.WindowingHiveCliDriver.main(WindowingHiveCliDriver.groovy:221) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:197)Caused by: java.lang.NumberFormatException: null at java.lang.Integer.parseInt(Integer.java:417) at java.lang.Integer.parseInt(Integer.java:499) at java_lang_Integer$parseInt.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116) at com.sap.hadoop.windowing.cli.WindowingClient.readPort(WindowingClient.groovy:110) at com.sap.hadoop.windowing.cli.WindowingClient.connect(WindowingClient.groovy:117) ... 21 more
Date: Mon, 13 Feb 2012 17:37:30 -0800
From: [email protected]
To: [email protected]
Subject: Re: [SQLWindowing] can not start the windowCli (#11)He is talking about the hive metastore server.
But your stacktrace indicates that you didn't get that far.Can you check the following things
- Your .sh points to com.sap.hadoop.windowing-0.0.1-SNAPSHOT.jar
- your -w option points to com.sap.hadoop.windowing-0.0.1-SNAPSHOT-jar-with-dependencies.jar
Reply to this email directly or view it on GitHub:
#11 (comment)
from sqlwindowing.
it's not enough to use mySQL as a metastore. You need to use a metastore service.
this is done by starting a TCP daemon which listens on 9083 port.
it's done by : hive --service metastore
please check the above post with configs for hive-site.xml
from sqlwindowing.
- The script should be:
THISSERVICE=windowingCliexport SERVICE_LIST="${SERVICE_LIST}${THISSERVICE} "
windowingCli () { CLASS=com.sap.hadoop.windowing.WindowingHiveCliDriver JAR=${HIVE_LIB}/com.sap.hadoop.windowing-0.0.1-SNAPSHOT.jar exec $HADOOP jar $JAR$CLASS "$ @"}
windowingCli_help() { windowingCli "--help"}
Don't use the jar-with-dependencies in the script. - On the command line give the full path to the 'com.sap.hadoop.windowing-0.0.1-SNAPSHOT-jar-with-dependencies.jar'
- as Ameet says you have to start hive metastore service. I haven't tested windowingCli directly connecting to mysql.
Hey Yong, maybe a quick call might help with resolving these issues. Let me know if you want to do this & when. Send me a private message.
from sqlwindowing.
OK. I started again to try this software. It is a very good feature, and I really want to make it work in my environment.
I got your 2 jar files. Without dependence one is put in $HIVE_HOME/lib, With-dependence one is put in $MY_HOME/lib
Here is the error with as detail information I can give you:
yzhang@bsdhcp15755:~> ls -ls
0 lrwxrwxrwx 1 yzhang users 23 Mar 22 10:57 hadoop -> ./hadoop-0.20.2-cdh3u3/
4 drwxr-xr-x 17 yzhang users 4096 Mar 22 14:14 hadoop-0.20.2-cdh3u3
0 lrwxrwxrwx 1 yzhang users 20 Mar 22 10:57 hive -> ./hive-0.7.1-cdh3u3/
4 drwxr-xr-x 9 yzhang users 4096 Mar 22 10:56 hive-0.7.1-cdh3u3
yzhang@bsdhcp15755:> cd hive/bin/ext//hive/bin/ext> ls
yzhang@bsdhcp15755:
cli.sh help.sh hiveserver.sh hwi.sh jar.sh lineage.sh metastore.sh rcfilecat.sh util windowCli.sh
yzhang@bsdhcp15755:~/hive/bin/ext> more windowCli.sh
THISSERVICE=windowingCli
export SERVICE_LIST="${SERVICE_LIST}${THISSERVICE} "
windowingCli () {
CLASS=com.sap.hadoop.windowing.WindowingHiveCliDriver
JAR=${HIVE_LIB}/com.sap.hadoop.windowing-0.0.1-SNAPSHOT.jar
exec $HADOOP jar $JAR
}
windowingCli_help () {
windowingCli "--help"
}
yzhang@bsdhcp15755:~/hive/bin/ext> ls ~/hive/lib/com.sap.hadoop.windowing-0.0.1-SNAPSHOT.jar
/home/yzhang/hive/lib/com.sap.hadoop.windowing-0.0.1-SNAPSHOT.jar
yzhang@bsdhcp15755:~/hive/bin/ext> ls ~/lib/com.sap.hadoop.windowing-0.0.1-SNAPSHOT-jar-with-dependencies.jar
/home/yzhang/lib/com.sap.hadoop.windowing-0.0.1-SNAPSHOT-jar-with-dependencies.jar
yzhang@bsdhcp15755:~> hive --service windowingCli -w /home/yzhang/lib/com.sap.hadoop.windowing-0.0.1-SNAPSHOT-jar-with-dependencies.jar
Exception in thread "main" java.lang.NoClassDefFoundError: groovy/lang/GroovyObject
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:295)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.util.RunJar.main(RunJar.java:190)
Caused by: java.lang.ClassNotFoundException: groovy.lang.GroovyObject
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 16 more
I can see the class groovy/lang/GroovyObject is in /home/yzhang/lib/com.sap.hadoop.windowing-0.0.1-SNAPSHOT-jar-with-dependencies.jar file. Any idea why?
from sqlwindowing.
Yes, thanks for catching this.
You have to place the groovy jar in $HIVE_HOME/lib also.
Have updated the README.txt and uploaded the groovy jar
from sqlwindowing.
Same as Issue 14
from sqlwindowing.
Related Issues (20)
- npath observations HOT 1
- Persistent Partitions failing when creating Temp directory HOT 1
- Parse Error: line 1:15 required (...)+ loop did not match anything at input 'partition' in statement HOT 10
- java.lang.reflect.InvocationTargetException HOT 20
- For 0.2 release, the lag function does not work HOT 6
- Service not found HOT 3
- looks for abc.<hivetable> HOT 22
- job was killed,but process running.. HOT 1
- Error when starting windowCli HOT 3
- mismatched input '/tmp/hadhive/' HOT 7
- the temp table can not find HOT 8
- enhancement the insert syntax
- Unknown partition column Symbol HOT 4
- No signature of method HOT 15
- Support for non-default db HOT 1
- Windowing function output path syntax HOT 1
- download the test data
- about com.sap.hadoop.windowing-0.0.2-SNAPSHOT.jar HOT 3
- SQL Windowing doesnt work on Hive 0.10
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sqlwindowing.