Spark-sql result mixed with running status about emr-bootstrap-actions

[shenzhun]

show tables;

ods_wa_language_pv_uv_hourly_mini_m
ods_wa_order_channel_hourly_litb
2015-02-03 07:47:51,378 INFO [SparkListenerBus] scheduler.StatsReportListener (Logging.scala:logInfo(59)) - task result size:(count: 1, mean: 15891.000000, stdev: 0.000000, max: 15891.000000, min: 15891.000000)
ods_wa_order_channel_hourly_litb_m
2015-02-03 07:47:51,378 INFO [SparkListenerBus] scheduler.StatsReportListener (Logging.scala:logInfo(59)) - 0% 5% 10% 25% 50% 75% 90% 95% 100%
ods_wa_order_channel_hourly_mini
2015-02-03 07:47:51,378 INFO [SparkListenerBus] scheduler.StatsReportListener (Logging.scala:logInfo(59)) - 15.5 KB 15.5 KB 15.5 KB 15.5 KB 15.5 KB 15.5 KB 15.5 KB 15.5 KB 15.5 KB
ods_wa_order_channel_hourly_mini_m

[christopherbozeman]

This is an issue of the log4j settings and stderr output showing up on console.

A quick way to handle this would be just to redirect stderr to somewhere else:

MASTER=yarn-client ~/spark/bin/spark-sql 2>/tmp/stderr.log

or

MASTER=yarn-client ~/spark/bin/spark-sql 2>/dev/null

[shenzhun]

yeah, it's a good way to handle it. Thanks.