HiveQueryTask doesn't use output/isn't atomic about luigi

[jcrobak]

I think that hive does this automatically -- output is written to e.g.: /tmp/hive-joe/hive_2013-05-08_14-14-09_760_4221650999782652997 and then moved atomically into place at the end of the job. Are you seeing something different?

I just verified this on our cluster with an INSERT OVERWRITE DIRECTORY query...

[freider]

Ooops, my mistake, I just confirmed it works here too... But I still think the output-aware query prefix generation would be a nice to have (it could be optional/configurable). Think it would make writing hive queries a bit cleaner with less output specific queries. What do you think?

[Tarrasch]

I'm closing this issue.

• It has been inactive for +4 months
• It's not about luigi core, so not as many users are affected about this.
• The change seems quite big, it's unlikely to be sporadically picked up.
• The owner haven't responded or disappeared.
• I don't understand what this is issue is about.

Every open issue adds some clutter, and we try to make the issues fewer and make it easier for new collaborators to find. Currently we try to close any issue that meets the first checkbox + one other.

Feel free to reopen this issue at any point if you have the intent to continue to work this. :)