so I've been trying to think of the best way to so...
# daft-dev
c
so I've been trying to think of the best way to solve the implicit coersion of sql. in python, we usually want things to be pretty explicit (explicit coercion), but in SQL, its generally accepted that you can use values as long as they can be cast to the datatype. ex: in python
Copy code
date_df = daft.from_pydict({"date_str":["2020-01-01","2020-01-02","2020-01-03"]})
date_df = date_df.with_column("date", col("date_str").str.to_date("%Y-%m-%d"))

# we probably want the user to use an explicit date object, or a casted string
date_df.filter(col("date") == datetime.date(2020, 1, 1)).explain()
date_df.filter(col("date") == daft.lit("2020-01-01").str.to_date("%Y-%m-%d")).explain()
but in sql, implicit coercions are almost always allowed.
Copy code
where date = '2020-01-01'; -- implicit
where date = cast('2020-01-01' as date); --explicit
Was hoping others might have some ideas on how we can solve this.
one idea i had was introduce a context object onto the
LogicalPlanBuilder
with some additional context, such as
allow_implicit_coercions
kinda like how we have our
DaftExecutionConfig
I think a
DaftPlanningConfig
would also make a lot of sense.
oh, wait.. we already have a PlanningConfig. 🤔
j
Yes, the planning config is applied when constructing the plan/optimizations etc The execution config is passed into each task when stuff is running
c
I think we do allow some implicit coercions elsewhere, such as i64 == f64. So should we just change the logic to allow
str == date
comparisons as well?
here's a PR to allow the
str == date
implicit coercion. https://github.com/Eventual-Inc/Daft/pull/3337 I do think we should revisit at some point to have a config flag for allowing/disabling implicit coercions all together.
k
I think this is reasonable. Are there any examples of implicit conversions we want to do in SQL but would want to disallow in Dataframe?
c
I opened up a separate issue to disable all implicit conversions https://github.com/Eventual-Inc/Daft/issues/3338