< Sammy Sidhu> or < jay> could one of you take a look at thi Distributed Data Community #daft-dev

<@U041QSEF2H2> or <@U042126MG49> could one of you ...

Cory Grinstead

04/01/2025, 6:11 PM

@Sammy Sidhu or @jay could one of you take a look at this pr. https://github.com/Eventual-Inc/Daft/pull/4034 it adds a pretty significant change to our datatypes system by using subclasses to represent the different variants. This should be familiar for anyone coming from pyarrow or polars. They both represent their datatype system via subclasses.

👀 1

Cory Grinstead

04/01/2025, 6:55 PM

for additional context, I decided to go with the subclasses as it is the convention set by other libraries (pyarrow, polars, ...). I believe @R. C. Howell was suggesting keeping it as a sum type and exposing the attributes directly on

DataType

instead of subclassing. (and having it return an
AttributeException
if trying to access it on the wrong type). I'm fine with this approach too, but it does deviate quite a bit from other libraries. do either of you have a preference here @Sammy Sidhu @jay

Robert Howell

04/01/2025, 7:39 PM

> keeping it as a sum type and exposing the attributes directly on

DataType

instead of subclassing. (and having it return an
AttributeException
if trying to access it on the wrong type There are not mutually exclusive, but we should push all switching on the rust side. I don't mind if you want additional classes, just that we implement this sanely.

jay

04/02/2025, 12:49 AM

@Sammy Sidhu is taking a look

🙌 1

Sammy Sidhu

04/02/2025, 8:21 AM

Just left my 2c on the PR

2 Views

Open in Slack

Previous Next