Hello, im new to daft and was wondering is their a way to take a dataframe and write it to a single parquet file. I'll explain my usecase I'm reading a few parquet files and want to merge them(similar to how compaction works with iceberg) in the hopes of contributing the code back to daft

Hi! Currently daft writes files into a directory specified by the _*`root_dir`*_ parameter, and because Daft is distributed, it is not straightforward to write into a single file. We are actually developing a new local execution engine right now, and this functionality should be available when it is released!

You can use the new engine by setting your environment variable:

DAFT_RUNNER=native

:)

Thanks @jay and @Colin Ho, I will try it