Skip to content

Improve the performance of hash joins on Flotilla when inputs only have 1 partition #5127

@desmondcheongzx

Description

@desmondcheongzx

Is your feature request related to a problem?

Since Daft 0.5.10, Flotilla supports hash joins by repartitioning both sides and performing local joins. This brings about benefits (since Flotilla can now be used) but introduces overheads in cases where the shuffle is unnecessary (e.g., when inputs only have a single partition).

Describe the solution you'd like

Elide shuffles when inputs only have 1 partition.

Describe alternatives you've considered

No response

Additional Context

No response

Would you like to implement a fix?

No

Metadata

Metadata

Assignees

Labels

p1Important to tackle soon, but preemptable by p0perf

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions