Comments (7)
I'll try and reproduce this if I can. What version of Hadoop? I'm assuming Spark 1.2?
from spark-jetty-server.
nm, I see 2.4 and spark 1.2 in your comment above
from spark-jetty-server.
I was able to reproduce this.
from spark-jetty-server.
Let me know if you still have issues with this.
from spark-jetty-server.
Yes, fixed. Able to start jetty but the Spring application is not reachable. I get the following errors:
[INFO] Configuring Jetty for project: war
[INFO] Webapp source directory = /home/tbda/spark-jetty-server/war/src/main/webapp
[INFO] web.xml file = /home/tbda/spark-jetty-server/war/src/main/webapp/WEB-INF/web.xml
[INFO] Classes = /home/tbda/spark-jetty-server/war/target/classes
[INFO] Property spark.home=/opt/spark-1.2.0-bin-hadoop2.4 was set
[INFO] Property spark.scheduler.allocation.file=/home/tbda/spark-jetty-server/war/src/main/resources/defaultPoolConfig.xml was set
[INFO] Property spark.ui.port=4040 was set
[INFO] Logging to org.slf4j.impl.SimpleLogger(org.mortbay.log) via org.mortbay.log.Slf4jLog
[INFO] Context path = /test
[INFO] Tmp directory = determined at runtime
[INFO] Web defaults = org/mortbay/jetty/webapp/webdefault.xml
[INFO] Web overrides = none
[INFO] Webapp directory = /home/tbda/spark-jetty-server/war/src/main/webapp
[INFO] Starting jetty 6.1.10 ...
[INFO] jetty-6.1.10
[INFO] No Transaction manager found - if your webapp requires one, please configure one.
[INFO] Initializing Spring root WebApplicationContext
[INFO] Root WebApplicationContext: initialization started
[INFO] Refreshing Root WebApplicationContext: startup date [Wed Jan 21 13:52:51 IST 2015]; root of context hierarchy
[INFO] Loading XML bean definitions from ServletContext resource [/WEB-INF/applicationContext.xml]
[INFO] Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@56c4c5fd: defining beans [sparkContext,sparkSqlContext]; root of factory hierarchy
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
[WARNING] Your hostname, localhost.localdomain resolves to a loopback address: 127.0.0.1; using 192.168.226.130 instead (on interface eth0)
[WARNING] Set SPARK_LOCAL_IP if you need to bind to another address
[INFO] Changing view acls to: tbda
[INFO] Changing modify acls to: tbda
[INFO] SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(tbda); users with modify permissions: Set(tbda)
[INFO] Slf4jLogger started
[INFO] Starting remoting
[INFO] Remoting started; listening on addresses :[akka.tcp://[email protected]:53230]
[INFO] Successfully started service 'sparkDriver' on port 53230.
[INFO] Registering MapOutputTracker
[INFO] Registering BlockManagerMaster
[INFO] Created local directory at /tmp/spark-local-20150121135253-8b6e
[INFO] MemoryStore started with capacity 224.6 MB
[WARNING] Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[INFO] HTTP File server directory is /tmp/spark-a6f1b166-b599-4477-989a-4d4946b14f42
[INFO] Starting HTTP Server
[INFO] jetty-8.y.z-SNAPSHOT
[INFO] Started [email protected]:51578
[INFO] Successfully started service 'HTTP file server' on port 51578.
[INFO] jetty-8.y.z-SNAPSHOT
[INFO] Started [email protected]:4040
[INFO] Successfully started service 'SparkUI' on port 4040.
[INFO] Started SparkUI at http://192.168.226.130:4040
[INFO] Added JAR /home/tbda/spark-jetty-server/services/target/services-0.0.1-SNAPSHOT.jar at http://192.168.226.130:51578/jars/services-0.0.1-SNAPSHOT.jar with timestamp 1421828574525
[INFO] Created pool default, schedulingMode: FAIR, minShare: 2, weight: 1
[INFO] Connecting to ResourceManager at /0.0.0.0:8032
[INFO] Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
[INFO] Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
[INFO] Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
[INFO] Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
Does this application require Hadoop daemons (specifically YARN) to be started?
from spark-jetty-server.
Yes it does and this is something I should probably add to the documentation. The purpose of this application is to submit a yarn job in Spark's yarn-client mode and expose the resulting SparkContext as a web service.
from spark-jetty-server.
Thanks ! Have issues with my Hadoop environment and will get back on how I progressed.
from spark-jetty-server.
Related Issues (2)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spark-jetty-server.