Ganglia stops running with 1.3.0.d on ami 3.6.0 (but not 3.5.0) about emr-bootstrap-actions

[christopherbozeman]

It does not appear to be an AMI issue, I can replicate the error in AMI 3.5.0 and 3.6.0. Looks to be more of Spark 1.3.0 issue given Spark 1.2.1 works without issue. It does take running a Spark app in yarn-cluster mode to trigger the problem.

May be related to https://issues.apache.org/jira/browse/SPARK-6484?

[jkleckner]

Sounds plausible. We're using 1.2.1 but try out our tests on the latest EMR just to be aware of issues.
If you roll a version with the patch, I would be happy to try it out.

[d18s]

It does take running a Spark app in yarn-cluster mode to trigger the problem.

I've just seen the same problem running an app in yarn-cluster mode (with Spark 1.3.0.d + AMI 3.6), haven't tested yarn-client mode

[jkleckner]

Bump.

[d18s]

I've just seen the same problem running an app in yarn-cluster mode (with Spark 1.3.0.d + AMI 3.6), haven't tested yarn-client mode

Seeing Ganglia stop running when running an app with yarn-client mode as well

[ericeijkelenboom]

Workaround: don't use the -g flag. This brings back Ganglia on a standalone cluster. Obviously, without the Spark specific metrics.

[christopherbozeman]

Confirmed this is related to https://issues.apache.org/jira/browse/SPARK-6484.

Created workaround in build 1.3.1.c, released with #97

[jkleckner]

Agreed. Tried it out and didn't see the problem, thanks.