Hey guys, I see that there's a `df.sample()` api t...
# general
n
Hey guys, I see that there's a
df.sample()
api to get a fractional sample of the dataframe. Is there an API that I can pass an integer to (ie. number of rows)? I just want to materialize n-random-rows as a daft dataframe, not show them in a notebook.
c
Hey @Neil Wadhvana, currrently theres no built in method to do this. If you're alright to sacrifice some efficiency, you can do a
df.count_rows()
and then sample with
df.sample(fraction=desired_num_rows/num_total_rows
. Also, I believe @Desmond Cheong is working on implementing sampling by
max_rows
,https://github.com/Eventual-Inc/Daft/issues/3332 .
n
That's exactly what I'm doing right now! Cool, thanks for the heads up!