New Feature Request: Handling retry at the notebook level about astro-provider-databricks

[tatiana]

Hi, @RafaelCartenet, this is a handy feature.
We could also have this associated with the Airflow task retry, so Airflow could automatically retry tasks on failure.
Would you be interested in contributing to the project?

[tatiana]

Hi @RafaelCartenet !

Those are great questions. Astronomer is contributing this provider to the Apache Airflow Databricks provider so it can be part of a larger community - both users and contributors.

These are the steps:

1. Contribute DatabricksNotebookOperator to Airflow Databricks provider ✅ (apache/airflow#39178)
2. Contribute DatabricksWorkflowTaskGroup to Airflow Databricks provider 🚧
3. Contribute DatabricksTaskOperator to Airflow Databricks provider
4. Contribute the Astro Provider Databricks plugin to Airflow Databricks provider
5. Add deferrable support for donated Databricks operators 🚧 (apache/airflow#39295)
6. Mark deprecation for astro-provider-databricks repo

I see the feature you requested being developed / contributed after the migration.

[RafaelCartenet]

Hey Tatiana I could eventually, could you please let me know what it takes to contribute ? Also I'm curious about the current status of this project. It's really promising though it doesn't look like being updated really often. Is Astronomer still looking into maintaining it long term ? Best Rafael

…

On Thu, May 9, 2024 at 10:29 PM Tatiana Al-Chueyr ***@***.***> wrote: Hi, @RafaelCartenet <https://github.com/RafaelCartenet>, this is a handy feature. We could also have this associated with the Airflow task retry, so Airflow could automatically retry tasks on failure. Would you be interested in contributing to the project? — Reply to this email directly, view it on GitHub <#65 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADXKHJRZYWF5MMDSFE26EHDZBOB6NAVCNFSM6AAAAABERXUJPWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBSG43TOMZSHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>