👋 Hi everyone! Thanks for making daft, I've been having a great time trying it out. Ran into a little bit of a snag today with inference using the ray backend though. It looks like it is re-initializing the stateful udf for every batch. Is there any way to force it to only perform initialization once to avoid the overhead of pulling the model and re-hydrating it every batch?

Hi Tyler! Just to make sure, are you setting the

DAFT_ENABLE_ACTOR_POOL_PROJECTIONS

environment variable? Our stateful UDFs currently are gated behind that config

Ah no I am not! I'll try that. Do I need to set it in the

ray.init

env vars as well or only from the driver process?

If you're running on ray, you can just set it in the Python script of your Ray job. Something like

import os
os.environ["DAFT_ENABLE_ACTOR_POOL_PROJECTIONS"] = "1"

import daft
# your code here

Sweet, giving that a go now.

With that flag set, you will also need to specify a concurrency that you'd like your stateful UDFs to run with. You can do this with

udf = udf.with_concurrency(n)

Awesome. That's great news. Will let you know how it goes.

Great to hear!