pentaho / kettle-storm Goto Github PK

Hi there -

The company I'm contracting for is interested in this project. I spent a bit of time getting it to work, but unfortunately I could not run the released .jar from the CI server. I then pulled the source and tried to run my simple ".ktr" transformation. I ran into some maven dependency issues (missing classes) which i worked around by hacking the pom.xml (The diffs from my pom.xml hacks are attached in case they are useful to you.)

After I got past the dependency errors I got the run time error reproduced below

My .ktr transformation ALMOST ran successfully.. It created the output file. but alas there was nothing in it.

Hope we can get this working... Deploying into a storm cluster would be very useful for us. If you need more info, pls let me know.

Thanks !

• chris

ERRORS I GOT WHEN RUNNING RELEASE .jar

jar=/home/chris/Downloads/kettle-engine-storm-0.0.1-SNAPSHOT-assembly.jar
java -Dkettle-storm-local-mode=true -jar $jar ~/esper/kettle/tutorial/hello.ktr
java.lang.NoClassDefFoundError: org/mozilla/javascript/JavaScriptException
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:190)
at org.pentaho.di.core.plugins.PluginRegistry.loadClass(PluginRegistry.java:368)
at org.pentaho.di.core.plugins.PluginRegistry.loadClass(PluginRegistry.java:308)
at org.pentaho.di.trans.step.StepMeta.(StepMeta.java:281)
at org.pentaho.di.trans.TransMeta.loadXML(TransMeta.java:3041)
at org.pentaho.di.trans.TransMeta.(TransMeta.java:2814)
at org.pentaho.di.trans.TransMeta.(TransMeta.java:2775)
at org.pentaho.di.trans.TransMeta.(TransMeta.java:2760)
at org.pentaho.di.trans.TransMeta.(TransMeta.java:2745)
at org.pentaho.di.trans.TransMeta.(TransMeta.java:2719)
at org.pentaho.di.trans.TransMeta.(TransMeta.java:2691)
at org.pentaho.kettle.engines.storm.StormExecutionEngine.init(StormExecutionEngine.java:82)
at org.pentaho.kettle.engines.storm.KettleStorm.main(KettleStorm.java:50)

RUN TIME ERROR AFTER BUILDING FROM SOURCE (using modified pom.xml)

INFO 05-11 15:23:17,258 - Session establishment complete on server localhost/127.0.0.1:2000, sessionid = 0x1422a93bc000010, negotiated timeout = 15000
INFO 05-11 15:23:17,273 - Processed session termination for sessionid: 0x1422a93bc000010
INFO 05-11 15:23:17,274 - Closed socket connection for client /127.0.0.1:53022 which had sessionid 0x1422a93bc000010
INFO 05-11 15:23:17,275 - Session: 0x1422a93bc000010 closed
INFO 05-11 15:23:17,275 - EventThread shut down
WARN 05-11 15:23:17,276 - Unable to process signal.
java.lang.IllegalStateException: Calling Context.exit without previous Context.enter
at org.mozilla.javascript.Context.exit(Context.java:452)
at org.pentaho.di.trans.steps.scriptvalues_mod.ScriptValuesMod.processRow(ScriptValuesMod.java:705)
at org.pentaho.kettle.engines.storm.bolt.KettleStepBolt.processRows(KettleStepBolt.java:176)
at org.pentaho.kettle.engines.storm.bolt.KettleStepBolt.onSignal(KettleStepBolt.java:274)
at backtype.storm.contrib.signals.AbstractSignalConnection.process(AbstractSignalConnection.java:44)
at com.netflix.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:38)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
INFO 05-11 15:23:25,883 - Setting new assignment for topology id tutorial-840b986e-33f6-4705-ae5b-23dc2becc2a9-1-1383693795: #backtype.storm.daemon.common.Assignment{:master-code-dir "/tmp/d760e317-f331-4c01-b08c-5aa740832278/nimbus/stormdist/tutorial-840b986e-33f6-4705-ae5b-23dc2becc2a9-1-1383693795", :node->host {"60afbde3-a1e0-4a3b-9976-0477dd6120ef" "ubuntu"}, :executor->node+port {[3 3] ["60afbde3-a1e0-4a3b-9976-0477dd6120ef" 4], [4 4] ["60afbde3-a1e0-4a3b-9976-0477dd6120ef" 4], [2 2] ["60afbde3-a1e0-4a3b-9976-0477dd6120ef" 4], [1 1] ["60afbde3-a1e0-4a3b-9976-0477dd6120ef" 4]}, :executor->start-time-secs {[4 4] 1383693805, [1 1] 1383693805, [2 2] 1383693805, [3 3] 1383693805}}

POM.xml changes (that allowed me to run the project after building from source):

diff --git a/kettle-engine-storm/pom.xml b/kettle-engine-storm/pom.xml
index 01739d0..f272a64 100644
--- a/kettle-engine-storm/pom.xml
+++ b/kettle-engine-storm/pom.xml
@@ -34,14 +34,26 @@
pentaho
http://repo.pentaho.org/artifactory/repo/

•    <repository>
  

•        <id>com.springsource.repository.bundles.release</id>
  

•        <name>EBR Spring Release Repository</name>
  

•        <url>http://repository.springsource.com/maven/bundles/release</url>
  

•    </repository>
  

•    <repository>
  

•        <id>com.springsource.repository.bundles.external</id>
  

•        <name>EBR External Release Repository</name>
  

•        <url>http://repository.springsource.com/maven/bundles/external</url>
  

•    </repository>
  

• storm storm ${storm.version}

•  <scope>provided</scope>
  

  @@ -50,6 +62,20 @@ ${storm.signals.version}

•  <groupId>rhino</groupId>
  

•  <artifactId>js</artifactId>
  

•  <version>1.7R2</version> 
  

•  <groupId>javax.mail</groupId>
  

•  <artifactId>com.springsource.javax.mail</artifactId>
  

•  <version>1.4.1</version> 
  

pentaho-kettle kettle-engine