I just got read_deltalake and write_deltalake with...
# general
j
I just got read_deltalake and write_deltalake with databricks unity catalog on azure working. Curious if there's any way yet to write into a managed table and or create a new managed table?
j
Creation is tricky… we are looking very seriously at building a catalog/table abstraction in Daft to be able to do this over Glue, Unity etc. any thoughts here on API?
j
The api in DataCatalogTable looks pretty good to me. I almost thought, hey, maybe this'll work and tried it anyway. Would likely need a table_provider param somehow.
j
Yeah the trickiest part is getting this to work across “all” catalogs and table formats. We’ll cook something up and get your thoughts!
j
Yeah for sure. The way I was thinking about it is if table=string, it's a local path, if it's of type DataCatalogTable, it's a path to a catalog provider location. Then DataCatalogType would need to be used to handle implementation. Complicated for sure though.