Skip to content

Conversation

gopidesupavan
Copy link
Member

closes #51005

When task killed externally the task process getting terminated immediately and no way to trigger on_kill functionality.

Registered on_kill to signal , this way when sigterm requested , this will trigger on_kill.

I dont have any other ideas to make it work. feel free to suggest if any other options available

before:

image

after:

image

example dag:


from time import sleep
from typing import TYPE_CHECKING

from airflow.sdk import BaseOperator
import time
from airflow.sdk.definitions.context import Context


class CustomOperator(BaseOperator):
    def __init__(self, **kwargs) -> None:
        super().__init__(**kwargs)

    def pre_execute(self, context):
        print("pre Execute!")

    def execute(self, context: Context):
        for i in range(10):
            print(f"Iteration {i}: Sleeping for 20 seconds...")
            time.sleep(20)

    def on_kill(self) -> None:
        self.log.info("on_kill called!")

import os
from datetime import timedelta
from airflow.sdk import DAG


with DAG(
    dag_id="check_on_kill",
    schedule=None,
    catchup=False,
) as dag:
    test_on_kill_hook = CustomOperator(
        task_id="test_on_kill_hook",
    )

    test_on_kill_hook




^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@gopidesupavan
Copy link
Member Author

dont have any other idea, feel free to suggest any other approach if this causes any sideffects

@gopidesupavan gopidesupavan force-pushed the fix-onkill-execution branch from b3a7e9b to ed13436 Compare July 24, 2025 16:05
@gopidesupavan gopidesupavan added the backport-to-v3-0-test Mark PR with this label to backport to v3-0-test branch label Jul 24, 2025
@eladkal eladkal added this to the Airflow 3.0.4 milestone Jul 27, 2025
@eladkal eladkal added the type:bug-fix Changelog: Bug Fixes label Jul 27, 2025
@gopidesupavan
Copy link
Member Author

can i have some review's on this please?

@gopidesupavan gopidesupavan merged commit d506349 into apache:main Jul 28, 2025
76 checks passed
Copy link

Backport failed to create: v3-0-test. View the failure log Run details

Status Branch Result
v3-0-test Commit Link

You can attempt to backport this manually by running:

cherry_picker d506349 v3-0-test

This should apply the commit to the v3-0-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

gopidesupavan added a commit to gopidesupavan/airflow that referenced this pull request Jul 28, 2025
* Make BaseOperator on_kill functionality work with TaskSDK

* Fix static checks

* Resolve review comments

* Update task-sdk/tests/task_sdk/execution_time/test_supervisor.py

Co-authored-by: Ash Berlin-Taylor <[email protected]>

---------

Co-authored-by: Ash Berlin-Taylor <[email protected]>
gopidesupavan added a commit that referenced this pull request Jul 29, 2025
…53832)

* Make BaseOperator on_kill functionality work with TaskSDK

* Fix static checks

* Resolve review comments

* Update task-sdk/tests/task_sdk/execution_time/test_supervisor.py



---------

Co-authored-by: Ash Berlin-Taylor <[email protected]>
RoyLee1224 pushed a commit to RoyLee1224/airflow that referenced this pull request Jul 31, 2025
* Make BaseOperator on_kill functionality work with TaskSDK

* Fix static checks

* Resolve review comments

* Update task-sdk/tests/task_sdk/execution_time/test_supervisor.py

Co-authored-by: Ash Berlin-Taylor <[email protected]>

---------

Co-authored-by: Ash Berlin-Taylor <[email protected]>
ferruzzi pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Aug 7, 2025
* Make BaseOperator on_kill functionality work with TaskSDK

* Fix static checks

* Resolve review comments

* Update task-sdk/tests/task_sdk/execution_time/test_supervisor.py

Co-authored-by: Ash Berlin-Taylor <[email protected]>

---------

Co-authored-by: Ash Berlin-Taylor <[email protected]>
fweilun pushed a commit to fweilun/airflow that referenced this pull request Aug 11, 2025
* Make BaseOperator on_kill functionality work with TaskSDK

* Fix static checks

* Resolve review comments

* Update task-sdk/tests/task_sdk/execution_time/test_supervisor.py

Co-authored-by: Ash Berlin-Taylor <[email protected]>

---------

Co-authored-by: Ash Berlin-Taylor <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:task-sdk backport-to-v3-0-test Mark PR with this label to backport to v3-0-test branch type:bug-fix Changelog: Bug Fixes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

on_kill Hook not executed on task stop
3 participants