MR mode - javax.jdo.JDOFatalInternalException about sqlwindowing

[hbutani]

Hi Ameet,

Can you send me more details on the error: stacktrace etc.
Which version of Hive are you using?
Also once you started the metastore server, can you connect to it via the regular hive cli.

regards,
Harish.

On 01/27/2012 01:42 PM, achaubal wrote:

Hi,

I downloaded the jar with dependencies : com.sap.hadoop.windowing-0.0.1-SNAPSHOT-jar-with-dependencies.jar
modified the cli.sh and copied it to windowingCli.sh
I can start the service by:
hive --service windowingCli -w /tmp/windowing/com.sap.hadoop.windowing-0.0.1-SNAPSHOT-jar-with-dependencies.jar
but when I type any command such as show databases, it errors out:
hive> show databases;
FAILED: Error in metadata: javax.jdo.JDOFatalInternalException: Unexpected exception caught.
NestedThrowables:
java.lang.reflect.InvocationTargetException
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

I have also started the metastore as described in the pdf but no dice.

thanks for your help

ameet

---

Reply to this email directly or view it on GitHub:
#1

[achaubal]

hi Harish,
thx for the reply.

I started the metastore server on the same machine as :
hive --service metastore
and can see that it's listening on 9083
I can run hive -p 9083 -h myhost and it lands me on a prompt
[myhost:9083] hive> show databases;
[Thrift Error]: Invalid method name: 'execute'
[Thrift Error]: Hive server is not cleaned due to thrift exception: Invalid method name: 'clean'

Here is the stacktrace. of the command:
hive --service windowingCli -w /tmp/windowing/com.sap.hadoop.windowing-0.0.1-SNAPSHOT-jar-with-dependencies.jar -hiveconf hive.root.logger=DEBUG,console

---

show databases;
12/01/27 17:19:54 DEBUG parse.VariableSubstitution: Substitution is on: show databases
12/01/27 17:19:54 INFO parse.ParseDriver: Parsing command: show databases
12/01/27 17:19:54 INFO parse.ParseDriver: Parse Completed
12/01/27 17:19:55 DEBUG fs.FileSystem: Creating filesystem for file:///
12/01/27 17:19:55 INFO ql.Driver: Semantic Analysis Completed
12/01/27 17:19:55 DEBUG lazy.LazySimpleSerDe: org.apache.hadoop.hive.serde2.DelimitedJSONSerDe initialized with: columnNames=[] columnTypes=[] separator=[[B@61b59919] nullstring= lastColumnTakesRest=false
12/01/27 17:19:55 DEBUG lazy.LazySimpleSerDe: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe initialized with: columnNames=[database_name] columnTypes=[string] separator=[[B@5570984c] nullstring= lastColumnTakesRest=false
12/01/27 17:19:55 INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from deserializer)], properties:null)
12/01/27 17:19:55 DEBUG lazy.LazySimpleSerDe: org.apache.hadoop.hive.serde2.DelimitedJSONSerDe initialized with: columnNames=[] columnTypes=[] separator=[[B@5a9bc984] nullstring= lastColumnTakesRest=false
12/01/27 17:19:55 INFO ql.Driver: Starting command: show databases
12/01/27 17:19:55 INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
12/01/27 17:19:55 DEBUG metastore.ObjectStore: Overriding datanucleus.cache.level2.type value null from jpox.properties with SOFT
12/01/27 17:19:55 DEBUG metastore.ObjectStore: Overriding datanucleus.plugin.pluginRegistryBundleCheck value null from jpox.properties with LOG
12/01/27 17:19:55 DEBUG metastore.ObjectStore: Overriding javax.jdo.option.ConnectionURL value null from jpox.properties with jdbc:derby:;databaseName=/var/lib/hive/metastore/metastore_db;create=true
12/01/27 17:19:55 DEBUG metastore.ObjectStore: Overriding datanucleus.autoStartMechanismMode value null from jpox.properties with checked
12/01/27 17:19:55 DEBUG metastore.ObjectStore: Overriding datanucleus.validateConstraints value null from jpox.properties with false
12/01/27 17:19:55 DEBUG metastore.ObjectStore: Overriding datanucleus.autoCreateSchema value null from jpox.properties with true
12/01/27 17:19:55 DEBUG metastore.ObjectStore: Overriding datanucleus.cache.level2 value null from jpox.properties with false
12/01/27 17:19:55 DEBUG metastore.ObjectStore: Overriding javax.jdo.option.ConnectionUserName value null from jpox.properties with APP
12/01/27 17:19:55 DEBUG metastore.ObjectStore: Overriding javax.jdo.option.DetachAllOnCommit value null from jpox.properties with true
12/01/27 17:19:55 DEBUG metastore.ObjectStore: Overriding datanucleus.connectionPoolingType value null from jpox.properties with DBCP
12/01/27 17:19:55 DEBUG metastore.ObjectStore: Overriding datanucleus.storeManagerType value null from jpox.properties with rdbms
12/01/27 17:19:55 DEBUG metastore.ObjectStore: Overriding javax.jdo.option.ConnectionDriverName value null from jpox.properties with org.apache.derby.jdbc.EmbeddedDriver
12/01/27 17:19:55 DEBUG metastore.ObjectStore: Overriding datanucleus.transactionIsolation value null from jpox.properties with read-committed
12/01/27 17:19:55 DEBUG metastore.ObjectStore: Overriding datanucleus.validateColumns value null from jpox.properties with false
12/01/27 17:19:55 DEBUG metastore.ObjectStore: Overriding javax.jdo.option.NonTransactionalRead value null from jpox.properties with true
12/01/27 17:19:55 DEBUG metastore.ObjectStore: Overriding datanucleus.validateTables value null from jpox.properties with false
12/01/27 17:19:55 DEBUG metastore.ObjectStore: Overriding javax.jdo.PersistenceManagerFactoryClass value null from jpox.properties with org.datanucleus.jdo.JDOPersistenceManagerFactory
12/01/27 17:19:55 DEBUG metastore.ObjectStore: Overriding datanucleus.identifierFactory value null from jpox.properties with datanucleus
12/01/27 17:19:55 DEBUG metastore.ObjectStore: datanucleus.autoStartMechanismMode = checked
12/01/27 17:19:55 DEBUG metastore.ObjectStore: datanucleus.identifierFactory = datanucleus
12/01/27 17:19:55 DEBUG metastore.ObjectStore: datanucleus.transactionIsolation = read-committed
12/01/27 17:19:55 DEBUG metastore.ObjectStore: datanucleus.validateTables = false
12/01/27 17:19:55 DEBUG metastore.ObjectStore: javax.jdo.option.ConnectionURL = jdbc:derby:;databaseName=/var/lib/hive/metastore/metastore_db;create=true
12/01/27 17:19:55 DEBUG metastore.ObjectStore: javax.jdo.option.DetachAllOnCommit = true
12/01/27 17:19:55 DEBUG metastore.ObjectStore: javax.jdo.option.NonTransactionalRead = true
12/01/27 17:19:55 DEBUG metastore.ObjectStore: datanucleus.validateConstraints = false
12/01/27 17:19:55 DEBUG metastore.ObjectStore: javax.jdo.option.ConnectionDriverName = org.apache.derby.jdbc.EmbeddedDriver
12/01/27 17:19:55 DEBUG metastore.ObjectStore: javax.jdo.option.ConnectionUserName = APP
12/01/27 17:19:55 DEBUG metastore.ObjectStore: datanucleus.validateColumns = false
12/01/27 17:19:55 DEBUG metastore.ObjectStore: datanucleus.cache.level2 = false
12/01/27 17:19:55 DEBUG metastore.ObjectStore: datanucleus.plugin.pluginRegistryBundleCheck = LOG
12/01/27 17:19:55 DEBUG metastore.ObjectStore: datanucleus.cache.level2.type = SOFT
12/01/27 17:19:55 DEBUG metastore.ObjectStore: javax.jdo.PersistenceManagerFactoryClass = org.datanucleus.jdo.JDOPersistenceManagerFactory
12/01/27 17:19:55 DEBUG metastore.ObjectStore: datanucleus.autoCreateSchema = true
12/01/27 17:19:55 DEBUG metastore.ObjectStore: datanucleus.storeManagerType = rdbms
12/01/27 17:19:55 DEBUG metastore.ObjectStore: datanucleus.connectionPoolingType = DBCP
12/01/27 17:19:55 INFO metastore.ObjectStore: ObjectStore, initialize called
FAILED: Error in metadata: javax.jdo.JDOFatalInternalException: Unexpected exception caught.
NestedThrowables:
java.lang.reflect.InvocationTargetException
12/01/27 17:19:55 ERROR exec.FetchTask: FAILED: Error in metadata: javax.jdo.JDOFatalInternalException: Unexpected exception caught.
NestedThrowables:
java.lang.reflect.InvocationTargetException
org.apache.hadoop.hive.ql.metadata.HiveException: javax.jdo.JDOFatalInternalException: Unexpected exception caught.
NestedThrowables:
java.lang.reflect.InvocationTargetException
at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:953)
at org.apache.hadoop.hive.ql.exec.DDLTask.showDatabases(DDLTask.java:1668)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:284)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1063)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:900)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:748)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:209)
at org.apache.hadoop.hive.cli.CliDriver$processCmd.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
at com.sap.hadoop.windowing.WindowingHiveCliDriver.processCmd(WindowingHiveCliDriver.groovy:148)
at com.sap.hadoop.windowing.WindowingHiveCliDriver$processCmd.callCurrent(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:46)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:133)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:141)
at com.sap.hadoop.windowing.WindowingHiveCliDriver.processLine(WindowingHiveCliDriver.groovy:85)
at com.sap.hadoop.windowing.WindowingHiveCliDriver$processLine.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
at com.sap.hadoop.windowing.WindowingHiveCliDriver.main(WindowingHiveCliDriver.groovy:255)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
Caused by: javax.jdo.JDOFatalInternalException: Unexpected exception caught.
NestedThrowables:
java.lang.reflect.InvocationTargetException
at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1186)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:803)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:698)
at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:234)
at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:261)
at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:196)
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:171)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:354)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.executeWithRetry(HiveMetaStore.java:306)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:451)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:232)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.(HiveMetaStore.java:197)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:108)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:1868)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:1878)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:951)
... 28 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at javax.jdo.JDOHelper$16.run(JDOHelper.java:1958)
at java.security.AccessController.doPrivileged(Native Method)
at javax.jdo.JDOHelper.invoke(JDOHelper.java:1953)
at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1159)
... 45 more
Caused by: java.lang.NullPointerException
at org.datanucleus.plugin.NonManagedPluginRegistry.registerBundle(NonManagedPluginRegistry.java:443)
at org.datanucleus.plugin.NonManagedPluginRegistry.registerBundle(NonManagedPluginRegistry.java:355)
at org.datanucleus.plugin.NonManagedPluginRegistry.registerExtensions(NonManagedPluginRegistry.java:215)
at org.datanucleus.plugin.NonManagedPluginRegistry.registerExtensionPoints(NonManagedPluginRegistry.java:156)
at org.datanucleus.plugin.PluginManager.registerExtensionPoints(PluginManager.java:82)
at org.datanucleus.OMFContext.(OMFContext.java:156)
at org.datanucleus.OMFContext.(OMFContext.java:137)
at org.datanucleus.ObjectManagerFactoryImpl.initialiseOMFContext(ObjectManagerFactoryImpl.java:132)
at org.datanucleus.jdo.JDOPersistenceManagerFactory.initialiseProperties(JDOPersistenceManagerFactory.java:363)
at org.datanucleus.jdo.JDOPersistenceManagerFactory.(JDOPersistenceManagerFactory.java:307)
at org.datanucleus.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:255)
at org.datanucleus.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:182)
... 53 more

12/01/27 17:19:55 DEBUG exec.DDLTask: org.apache.hadoop.hive.ql.metadata.HiveException: javax.jdo.JDOFatalInternalException: Unexpected exception caught.
NestedThrowables:
java.lang.reflect.InvocationTargetException
at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:953)
at org.apache.hadoop.hive.ql.exec.DDLTask.showDatabases(DDLTask.java:1668)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:284)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1063)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:900)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:748)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:209)
at org.apache.hadoop.hive.cli.CliDriver$processCmd.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
at com.sap.hadoop.windowing.WindowingHiveCliDriver.processCmd(WindowingHiveCliDriver.groovy:148)
at com.sap.hadoop.windowing.WindowingHiveCliDriver$processCmd.callCurrent(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:46)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:133)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:141)
at com.sap.hadoop.windowing.WindowingHiveCliDriver.processLine(WindowingHiveCliDriver.groovy:85)
at com.sap.hadoop.windowing.WindowingHiveCliDriver$processLine.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
at com.sap.hadoop.windowing.WindowingHiveCliDriver.main(WindowingHiveCliDriver.groovy:255)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
Caused by: javax.jdo.JDOFatalInternalException: Unexpected exception caught.
NestedThrowables:
java.lang.reflect.InvocationTargetException
at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1186)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:803)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:698)
at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:234)
at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:261)
at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:196)
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:171)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:354)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.executeWithRetry(HiveMetaStore.java:306)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:451)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:232)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.(HiveMetaStore.java:197)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:108)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:1868)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:1878)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:951)
... 28 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at javax.jdo.JDOHelper$16.run(JDOHelper.java:1958)
at java.security.AccessController.doPrivileged(Native Method)
at javax.jdo.JDOHelper.invoke(JDOHelper.java:1953)
at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1159)
... 45 more
Caused by: java.lang.NullPointerException
at org.datanucleus.plugin.NonManagedPluginRegistry.registerBundle(NonManagedPluginRegistry.java:443)
at org.datanucleus.plugin.NonManagedPluginRegistry.registerBundle(NonManagedPluginRegistry.java:355)
at org.datanucleus.plugin.NonManagedPluginRegistry.registerExtensions(NonManagedPluginRegistry.java:215)
at org.datanucleus.plugin.NonManagedPluginRegistry.registerExtensionPoints(NonManagedPluginRegistry.java:156)
at org.datanucleus.plugin.PluginManager.registerExtensionPoints(PluginManager.java:82)
at org.datanucleus.OMFContext.(OMFContext.java:156)
at org.datanucleus.OMFContext.(OMFContext.java:137)
at org.datanucleus.ObjectManagerFactoryImpl.initialiseOMFContext(ObjectManagerFactoryImpl.java:132)
at org.datanucleus.jdo.JDOPersistenceManagerFactory.initialiseProperties(JDOPersistenceManagerFactory.java:363)
at org.datanucleus.jdo.JDOPersistenceManagerFactory.(JDOPersistenceManagerFactory.java:307)
at org.datanucleus.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:255)
at org.datanucleus.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:182)
... 53 more

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

12/01/27 17:19:55 ERROR ql.Driver: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

[achaubal]

oh and the hive version is hive-hwi-0.7.1-cdh3u2.war , i.e. 0.7.1

[hbutani]

Can you make sure you have the following set in the configuration, when you run the hive cli:
hive.metastore.uris -> set to thrift://localhost:9083, or whatever ur server is
hive.metastore.local -> false

Make sure that the vanila hive cli is working; show databases etc.

From the stacktrace looks like the hive client is not talking to the metastore server.
Alos if you don't set 'hive.metastore.local' to false, the hive client will try to create a metastore_db on the filesystem from where you run; if this is the same place from you started the matstore server than bad things can happen. Check derby.log and see if there are any errors in there.

[achaubal]

ok, I was able to make the metastore service work fine. I split it across 2 machines, one running the service and the other running the client. my problem was passing -h and -p to the hive command line. all I had to do was to change the hive-site file to point to the metastore service machine.
now moving to step 2: and more problems :) in a separate thread