hi everyone, i need to transform a daft column to a python list like

<http://temp_df.col1.to|temp_df.col1.to>_list()

in pandas dataframe. How can i do that without transforming daft df to another data type ?

Hey! If I’m understanding this correctly, you’d like to have your column as a Python list? You can always retrieve your data like so:

df.select(“col1”).to_pydict()[“col1”]

Hi Jay, i was hope that i can find a alternative way for transforming a daft dataframe column to list but i guess i have to transform whole dataframe to dict data type.

Yes, unfortunately any implementation that we do would likely just be syntactic sugar on top of this 😛 Under the hood, we have to: 1. Prune all the other columns from each distributed shard of the dataframe 2. Collect the data from the

col1

column on your machine 3. Concat the data together 4. Export it as a Python list

things getting complicated when distributing data over cpus without indexing like pandas i guess 🙂

Yup the distributed computing makes it a little trickier than just having arrays/series like Pandas and Polars