@Cory Grinstead @Sammy Sidhu first PR on our path to non-equi joins -- unresolved columns! it's almost 2000 lines of new code 😄😵 lmk if you'd like me to walk you through it https://github.com/Eventual-Inc/Daft/pull/3804

For the column thing, how are sql expressions being handled? It seems like a similar issue to

col("a.b")

df.select(daft.sql_expr("a.b")).show()

In your example there’s no table to plan the expr against so the sql planner defaults to the column “a.b”. there are a few options for us to improve this: • have the default behavior be turning compound idents into struct gets • hold the sql expression unplanned until it gets associated with a table and plan it with the correct input table

More specifically, is

col("a.b")

equivalent to

sql_expr("a.b")

?

yes

Ok, this is what I was asking and suggesting by saying it's a similar issue which you've address on one side. I would advise against turning compound identifiers (identifier chains) into struct gets without a delimiter. It creates a syntax ambiguity which you'll see other implementations address with a path delimiter

:

so to do struct stuff you can have

a:b.c

which is the same as

col("a")."b"."c"

. This is to avoid the ambiguity when you have table-qualified identifiers.

Which engines require the ":" delimiter? I wanted to turn dots into struct gets because that's what DuckDB allows: https://duckdb.org/docs/sql/data_types/struct.html#dot-notation-order-of-operations

Spark

It looks like Spark allows dot for struct get

Nice, I believe the ':' is for path expressions on variant.

@Kevin Wang I left an initial review. I want to take a closer look at it on Tuesday though.

Thanks! Left some replies to your review comments. There's a lot of nuance to it so feel free to grab some time with me to talk through it.

yeah i think we should probably jump on a call about this. I still have quite a few questions about the

Column::Unresolved

variant.