Community for the Daft project and all things distributed data

Distributed Data Community

image.png

<@U042126MG49> I do some profiling of both "clean builds" and "incremental" (touching 1 file in daft-core)

For incremental builds, it seems like we have 2 issues.
1. a bunch of crates are waiting on on the frontend of `daft-core` to start compiling. (codegen of `daft-core`can be pipelined with the frontend compilation of downstream crates)
2. we have an insane amount of codegen in daft-core. Im guessing this is due to a ton of macros and generics. 


inc.html

full-build.html

Because I just modified a file in `daft-core` ,`daft-io` doesn't depend on `daft-core`  so it doesn't have to be recompiled

Here is what it looks like if I touch `daft-io`
and the command to run to produce the report yourself
```maturin develop --timings=html -Z unstable-options```