I think we can start advertising spark connect a bit more. We have pretty good support for ray + sql. The dataframe api is still missing a lot of ops, but it should run most sql queries.

Shall we get started on some documentation here? Definitely open to marking this as a beta feature and writing a page about it (cc @ChanChan Mao )

yes agree!

Yes both! Also would be great to have a little feature matrix thing so folks can have some expectation of what works/doesn’t work yet!