Btw @Colin Ho have the numbers for the local benchmark (

DAFT_RUNNER=native pytest tests/benchmarks/test_local_tpch.py -m benchmark

) increased dramatically for you? I think when I checked with Jay, our daily benchmarks don't show any slowdown, but I'm seeing anywhere from 4-12x speedups. It seems the more queries I test the bigger the slowdowns, so maybe this is related to something we did for concurrency?

I think daily benchmarks don’t run on native

Are u running on main? And what scale factor / number of files?

Yep, tried running on main, and just on the default 0.2 scale factor with 1-2 parts

hmmm interesting, so like if you test all 22 questions for example, there's a overall slowdown across the board vs if you just tested maybe like the first 10 questions?

yep. If I just test the default 1-11 it slowed down too

are you comparing current main vs some previous version/commit?

lol you know what it is... I had modified my command line script to use

make build

instead of

make build-release

for some reason 🤦‍♂️ still a little surprised that on the dev build the number of queries affected runtime, but I guess it was a red herring

Damn hahahah