Skip to content

Fix test_roundtrip_provider_example_dags #49647

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 23, 2025
Merged

Fix test_roundtrip_provider_example_dags #49647

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion airflow-core/tests/unit/serialization/test_dag_serialization.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -685,10 +685,14 @@ def validate_deserialized_dag(self, serialized_dag: DAG, dag: DAG):
for field in fields_to_check:
actual = getattr(serialized_dag, field)
expected = getattr(dag, field, None)

if field == "max_consecutive_failed_dag_runs":
# TaskSDK sets -1 default to max_consecutive_failed_dag_runs
assert actual in [expected, 0], f"{dag.dag_id}.{field} does not match"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ashb @amoghrajesh @kaxil -> just checking if that was intentional change to change it it to -1 ? max_consecutive_runs is experimental so should not matter, just checking

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merrging to unblock main without waiting

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really do not recall why we did this, maybe there was no good reason to do it at all.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it was -1 in the serialization side and set to default from config elsewhere? Or perhaps it was when we hadn't yet worked out what we would do for config in the task-sdk dist yet.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes the default value zero being set here https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/models/dag.py#L438. so should we align all with zero?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah

continue
assert actual == expected, f"{dag.dag_id}.{field} does not match"
# _processor_dags_folder is only populated at serialization time
# it's only used when relying on serialized dag to determine a dag's relative path
assert dag._processor_dags_folder is None
assert (
serialized_dag._processor_dags_folder
== (AIRFLOW_REPO_ROOT_PATH / "airflow-core" / "tests" / "unit" / "dags").as_posix()
Expand Down