Community for the Daft project and all things distributed data

Distributed Data Community

hey team! any document or design doc about daft-connect? apply daft logical-plan to spark connect?

Hey Chuanlei. daft-connect is still in pretty early development, so unfortunately we don't have much for docs on it yet.

We do have an <https://github.com/Eventual-Inc/Daft/issues/3581|issue in github> that's a good central point for daft-connect related work.  There is a section there for documentation work we want to do.

but at a very high level, yes we essentially convert the spark connect plan to a daft logical plan, then execute it the same as if it were written using daft.