How does daft decide which strategy to use for a j...
# general
t
How does daft decide which strategy to use for a join?
d
For our local executor, we just use a hash join. For non-local execution, we do the following: 1. If a join strategy has been specified, use that. 2. Check if we can broadcast the larger table against the following conditions: a. The larger side of the join is not already partitioned on the join key b. The smaller table is under the broadcast threshold size c. The broadcast side is not the "outer" side of a join (e.g. in an outer join, or the "outer" side of an anti/semi/left outer/right outer join) 3. If not, check if we can do a sort-merge join a. Is this an inner join b. Are the left and right join keys primitives c. Is the larger side range partitioned 4. If not, use a hash join
🙏 1