https://www.getdaft.io logo
Join Slack
Powered by
current Make file is failing if used with python 3...
# daft-dev
s
current Make file is failing if used with python 3.7 because package uv requires >= 3.8 .
anyone tried installing daft with 3.7 using requirement-dev.txt ? getting version error's .🫠
j
Ahhhhh hmm we were just discussing dropping 3.7 support entirely internally @Sammy Sidhu even more reason to do it?
👍 1
s
@jay from my understanding, @sherlockbeard is actually developing in 3.7!
😮 1
s
Nope nope @Sammy Sidhu developing in 3.10 . For testing and debugging tried creating 3.7
s
instead of running the makefile which runs 
uv
, you should also be able to run 
pip install -r requirements-dev.txt
Also I went ahead and ran your branch with a 3.7 setup and got the following error
Copy code
aws_server_ip = '127.0.0.1', aws_server_port = 5000, aws_log_file = <_io.TextIOWrapper name='/tmp/pytest-of-sammy/pytest-5/aws_logging0/aws_log.txt' mode='w' encoding='UTF-8'>

    @pytest.fixture(scope="session")
    def aws_server(aws_server_ip: str, aws_server_port: int, aws_log_file: io.IOBase) -> Iterator[str]:
        # NOTE(Clark): The background-threaded moto server tends to lock up under concurrent access, so we run a background
        # moto_server process.
        aws_server_url = f"http://{aws_server_ip}:{aws_server_port}"
        old_env = os.environ.copy()
        # Set required AWS environment variables before starting server.
        # Required to opt out of concurrent writing, since we don't provide a LockClient.
        os.environ["AWS_S3_ALLOW_UNSAFE_RENAME"] = "true"
        try:
            # Start moto server.
            process = start_service(aws_server_ip, aws_server_port, aws_log_file)
            yield aws_server_url
        finally:
            # Shutdown moto server.
>           stop_process(process)
E           UnboundLocalError: local variable 'process' referenced before assignment

tests/io/delta_lake/conftest.py:259: UnboundLocalError
============================================================================================================== short test summary info ==============================================================================================================
ERROR tests/io/delta_lake/test_table_read.py::test_deltalake_read_full[s3_path-1-unpartitioned-None] - UnboundLocalError: local variable 'process' referenced before assignment
Looks like something is broken in the fixture in 3.7?
s
@Sammy Sidhu https://dist-data.slack.com/archives/C052CA6Q9N1/p1713549878306439?thread_ts=1713549397.977209&cid=C052CA6Q9N1 Tried requirements-dev . it failed in ray install . System -> m1 Mac Python -> 3.7.17
@Sammy Sidhu I removed delta lake install for python 3.7 . we should skip these delta lake test for 3.7
s
Yeah 3.7 on m1 is a pain! I think we should just skip the tests if we're on 3.7
🤧 1
yeah exactly
Here's a diff that you can use to do so
👍 1
you prob to also do it for 
tests/io/delta_lake/test_table_read_pushdowns.py
s
Hmm . thanks for the diff maybe its docker time for python 3.7 and Ubuntu
s
Yeah sorry about this haha! We'll likely drop support for 3.7 in the next release
s
but we are doing deltalake = pytest.importorskip("deltalake") in all delta tests. If deltalake is not there which will be the case in github action is not there it should skip it .
ok it looks like it will skip everything
Copy code
pytest --ignore tests/integration  -r s
================================================================== test session starts ===================================================================
platform linux -- Python 3.7.17, pytest-7.4.3, pluggy-1.2.0
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /usr/Daft
configfile: pyproject.toml
testpaths: tests
plugins: cov-4.1.0, benchmark-4.0.0, hypothesis-6.79.2, lazy-fixture-0.6.3
collected 0 items / 1 skipped                                                                                                                            

================================================================ short test summary info =================================================================
SKIPPED [1] tests/io/delta_lake/conftest.py:25: could not import 'deltalake': No module named 'deltalake'
🔥 1
Open in Slack
PreviousNext
Powered by