I'm reading the Daft code. I want to sure whether the

daft.table.Table

python class still useful. It seems that no other module use this python class.

Hi! Our Table used to be the primary local abstraction, but is now encapsulated within a MicroPartition MicroPartitions are the “local” abstraction of a dataframe. When Dataframes get processed, they are done so in parallel across many MicroPartitions at a time. Under the hood, MicroPartitions can actually be represented as multiple Tables. But this is all in Rust now. Hope this helps!

thx for the reply. I see. It is confusing for the mirco-partition, since mirco-partition does not hold a

Table

member and these two classes have a lot of duplicated implementations at the operator level. May I help to refactor these code to remove

Table

python&rust class?

MicroPartition does hold a

Table

member! Check out the state object. The main difference is that MicroPartition is lazy — it can hold either a

TableState:Loaded

variant (which holds a

Vec<Table>

), or a

TableState::Unloaded

variant (which holds a ScanTask) This allows us to save some work for certain operations that might not need to materialize

Tables

at all! Hence it wouldn’t be possible to remove

Table

today, it’s pretty important to the way the system works

ok, thx, I missed the state member.