Is there an inbuilt way to show the progress of the writing? (i.e.

.write_deltalake()

) - with logging disabled ?

Hey @Ammar Bitar! We are working on better mechanisms for observability of a running query, but generally speaking writing doesn’t really happen as one step (it gets fused with other steps and ran together in a single task) For example, if your workload is a read -> filter -> write, then Daft will run tasks that look like

Scan -> Filter -> Write

and we report the completion of these tasks as a progress bar

Thanks for the reply! 🙂 Related, is there a way to quiet daft logging, without quieting the progress bar?

What kind of logging are you seeing that you’d like to quiet?