I am seeing some performance regressions in the latest versi Distributed Data Community #daft-dev

I am seeing some performance regressions in the la...

Matthew Powers

05/20/2025, 12:48 PM

I am seeing some performance regressions in the latest version of Daft, probably from the native engine. I think Daft is great and am just reporting these to make the team aware.

❤️ 3

Robert Howell

05/20/2025, 3:41 PM

Thanks for making us aware, could you please share a little bit more about the test setup like scale and runner used? Tagging @Colin Ho as well.

Colin Ho

05/20/2025, 8:53 PM

I believe its this: https://github.com/MrPowers/mrpowers-benchmarks ? I did try running the

run_all_groupby.py

benchmark on my local mac (36 gb ram, 12 cores), and the results i get are rather different. If you could share which machine you use, that'd be great for diagnosing if there's an issue with daft on specific platforms!

Srinivas Lade

05/20/2025, 11:35 PM

@Matthew Powers btw Q8 should work on Daft 0.4.15 with our recent window function additions. Tested and it worked

Matthew Powers

05/21/2025, 1:51 AM

@Srinivas Lade - sweet, will add q8 🤓

Matthew Powers

05/21/2025, 1:53 AM

@Colin Ho - Yea, that's right, I ran it on the 1e8 dataset. Here are my machine specs: macOS 14.5, 16GB memory, M3 chip

Matthew Powers

05/21/2025, 1:56 AM

I've seen this before with other engines. Sometimes query times spike when memory constraints start kicking in. You have 36 GB, so that's maybe why you didn't see them in q3. BTW, once we get these great, I think we should definitely add Daft to the official duckdb benchmarks: https://duckdblabs.github.io/db-benchmark/. It'll be great publicity!

💖 1

Open in Slack

Previous Next