Hey folks! I'm trying to add some test cases for my custom UDFs. I'm using the latest

daft

release to do this. When I use the

native

or

ray

runner, I get next to no test coverage on these UDFs, meaning that

pytest-cov

is not seeing the invoked lines. When I use the

py

runner, I get inconsistent coverage behavior. When randomizing the order of tests, I get 3 different coverage numbers (my module has 3 UDFs in it, so it seems

pytest

is only reporting the last-executed coverage. Do you have some testing docs I can refer to? The code base will soon contain a slew of custom UDFs, and I'd like a strategy to get unit test coverage before diving in.

hey @Neil Wadhvana! I think I have seen this before. I believe this is due to both ray and the native runner using separate processes for the UDF. This causes only the initial process to record the coverage. One workaround we have done is the following in daft's init.py https://github.com/Eventual-Inc/Daft/blob/main/daft/__init__.py#L8 This may be the culprit or it may be you need to enable this in your UDF. A question I have is how many .coverage files are you getting when you run the query. We may need to just merge them. you can use the

coverage combine

command for that

after i run pytest, i usually only have one .coverage file. I've also tried manually running combine but to no avail

I don't believe the native runner uses separate processes for UDFs except when concurrency is specified, but it does have a different threading behavior than the py runner. Unsure what would cause this issue. @Sammy Sidhu are there any more things Neil could try?

these are unit tests, so if there's a way to force them to run single-threaded i'm more than happy as well