depends_on_past at DAG level about airflow

[artwr]

@rijuk, What you are looking for is the parameter wait_for_downstream. A run for a task instance will then only happen if previous runs of the downstream tasks are complete.

[mistercrunch]

"A run for a task instance will then only happen if previous runs of the directly downstream tasks of that task are succesful."

If you want to force the completion of the entire DAG, you'll need to put a trigger (say you could land a dummy trigger file in HDFS or S3) at the very end of all the tasks in your DAG, and use a sensor at the very beginning of your DAG (waiting for that that trigger file to show up in HDFS or S3).

[rijuk]

Thanks.