Anyone experienced DispatchFailure(DispatchFailure { source: ConnectorError { kind: Timeout, source: HttpTimeoutError { kind: "HTTP read", duration: 30s }, connection: Unknown } }) before when reading from S3?

Hey @Kyle is this ephemeral? You might need to increase some of the retries Sometime S3 will scream at you if you hit it too hard in a short amount of time 🫡

Do you have a snippet?

Most of the times yes it just comes and goes but my latest ray job died after hundreds of this error

I would recommend bumping up the number of tries to something like 15-20 in the S3Config! https://www.getdaft.io/projects/docs/en/latest/api_docs/doc_gen/io_configs/daft.io.S3Config.html

Oh cool thank you!