We’re actually good friends with the folks over at Lance. Would be very curious to hear your thoughts on where that lands in terms of the landscape of tooling.
I think your hypothesis around centering around Ibis as an API is sound. We have heard from some folks that they do suffer from dataframe API fatigue, since there are so many options available now.
Our arrow support is zero-copy for non-Python DataTypes. Injecting those data structures into polars or duckdb is a good idea for scalar expression support, but we haven’t yet had the engineering hours required to execute that yet 😁