I refactored our PhysicalPlan for shuffles! cc @Sammy Sidhu @Colin Ho • Easier to extend and add new shuffle implementations • Also I think this is a little easier to understand from the users’ POV. No more

Flatten

,

FanoutHash

, or

ReduceMerge

😛 • Easy to draw query boundaries too if/when we need to understand where the pipeline breakers are (just look for the ShuffleExchanges) Found a bug (needed PhysicalPlan optimization) during the process (see the second join plan which has a redundant ShuffleExchange). The new PhysicalPlan makes it easier to spot these too because it’s no longer littered with chains of

FanoutHash -> ReduceMerge

Ready for us to go wild with the new shuffles…….