-
Notifications
You must be signed in to change notification settings - Fork 15.6k
Make BaseOperator on_kill functionality work with TaskSDK #53718
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make BaseOperator on_kill functionality work with TaskSDK #53718
Conversation
dont have any other idea, feel free to suggest any other approach if this causes any sideffects |
b3a7e9b
to
ed13436
Compare
can i have some review's on this please? |
Co-authored-by: Ash Berlin-Taylor <[email protected]>
Backport failed to create: v3-0-test. View the failure log Run details
You can attempt to backport this manually by running: cherry_picker d506349 v3-0-test This should apply the commit to the v3-0-test branch and leave the commit in conflict state marking After you have resolved the conflicts, you can continue the backport process by running: cherry_picker --continue |
* Make BaseOperator on_kill functionality work with TaskSDK * Fix static checks * Resolve review comments * Update task-sdk/tests/task_sdk/execution_time/test_supervisor.py Co-authored-by: Ash Berlin-Taylor <[email protected]> --------- Co-authored-by: Ash Berlin-Taylor <[email protected]>
…53832) * Make BaseOperator on_kill functionality work with TaskSDK * Fix static checks * Resolve review comments * Update task-sdk/tests/task_sdk/execution_time/test_supervisor.py --------- Co-authored-by: Ash Berlin-Taylor <[email protected]>
* Make BaseOperator on_kill functionality work with TaskSDK * Fix static checks * Resolve review comments * Update task-sdk/tests/task_sdk/execution_time/test_supervisor.py Co-authored-by: Ash Berlin-Taylor <[email protected]> --------- Co-authored-by: Ash Berlin-Taylor <[email protected]>
* Make BaseOperator on_kill functionality work with TaskSDK * Fix static checks * Resolve review comments * Update task-sdk/tests/task_sdk/execution_time/test_supervisor.py Co-authored-by: Ash Berlin-Taylor <[email protected]> --------- Co-authored-by: Ash Berlin-Taylor <[email protected]>
* Make BaseOperator on_kill functionality work with TaskSDK * Fix static checks * Resolve review comments * Update task-sdk/tests/task_sdk/execution_time/test_supervisor.py Co-authored-by: Ash Berlin-Taylor <[email protected]> --------- Co-authored-by: Ash Berlin-Taylor <[email protected]>
closes #51005
When task killed externally the task process getting terminated immediately and no way to trigger on_kill functionality.
Registered on_kill to signal , this way when sigterm requested , this will trigger on_kill.
I dont have any other ideas to make it work. feel free to suggest if any other options available
before:
after:
example dag:
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rst
or{issue_number}.significant.rst
, in airflow-core/newsfragments.