@Kyle hive partitioned reads, and hive partition pruning, are supported in the latest release! Simply add
hive_partitioning=True
to your`read_{parquet, csv, json}` commands. Do let us know if this works well for you :)
k
Kyle
11/06/2024, 11:12 PM
Great thank you!! I'll try it out!!
Kyle
11/07/2024, 8:18 AM
Hive partitioning reads work well thank you!
For the _*write_mode*_ in the write_parquet though I found that the docs has a small discrepancy in the parameters section which calls it mode instead of the correct _*write_mode*_ parameter.
d
Desmond Cheong
11/07/2024, 10:12 AM
Glad to hear it!
And thanks for the catch, will fix the docs tomorrow