Neil Wadhvana
11/18/2024, 9:53 PMdaft. These are packaged to support np.generic types which I run using pytest.parametrize() .
For the ray.data version of these transforms, I've written a simple session scope fixture which initializes a ray cluster with some CPU cores and uninitializes it at the end of the pytest run. This is used by a Test class, making it easy to trace in code.
For the daft test, I have an analogous Test class. I'd like to re-use this class and simply re-decorate it with another session or module scope fixture which sets and unsets the daft.context.runner.
How do? Has the context setting bug been resolved? Is there another way to handle this which you use for tests internally that you can point me to?
I would of course like to extend this to the new python streaming execution API as well, hopefully with another session or module scope fixture. But the ray runner is the highest priority right now.Desmond Cheong
11/19/2024, 3:57 AMIs there another way to handle this which you use for tests internally that you can point me to?One thing we've tried doing within tests is to set the runner to
None, which might accomplish what you want to do for the time being
https://github.com/Eventual-Inc/Daft/blob/274f300fb86413dfea11d38fba1fe93e7db2f02b/tests/actor_pool/test_actor_cuda_devices.py#L131
We'll probably try to have a better solution in the near future