For those of yall that missed it…

pip install daft-launcher

Initialize the launcher’s configs (default AWS region, number of machines etc)

daft init-config

Launch a cluster and see that it’s running

daft up
daft list

Connect to the cluster!

daft connect

Tear it down

daft down

It’s easier more than ever now to try Daft out on a distributed environment, all you need is an AWS account and use

daft-launcher

I got ModuleNotFoundError: No module named 'tomllib' after installing 🥲 I guess there is a python version dependency?

@Raunak Bhagat halp

Ah yes,

tomllib

is for >=3.11 and

toml

is for < 3.11 I believe... I'll have to see how to fix this. @Kyle Apologies for that mistake! If possible, are you able to use 3.11 for pure testing purposes? If not, I'll try and get a fix up asap.

No worries! I don't think I will be using the launcher for the time being because I don't have access to the cloud resources directly. Just wanted to test the package to see where I'd get stuck 😬

For sure, that makes sense. I've been trying to iron out all of the bugs, so your trial-and-error process is greatly appreciated!

@Kyle do yall use kubernetes? I wonder if maybe we should make a kubernetes backend for daft-launcher. It would require certain permissions though, such as the ability to launch pods in the provided namespace, and port-forward capabilities

Yes we do but I generally do not have the permissions to touch those configurations. It also wouldn't be a major concern for me at the moment. I think the questions you're asking on configuration settings in cloud could be useful as well if people knew what to ask for, so that they can determine and find out if there are some existing resources within their own companies they could reuse or if they need to be the ones setting it up.

Hey Kyle, forgot to update you earlier, but the toml problem has been fixed in v0.3.7! If you have the chance, please give it a try. Also, the minimum python version is set to 3.9.

Okay thanks! I was also wondering, is it possible to peer into an existing cluster to view how it was set up and then reuse all of those configs to down and up the cluster?

Hmm, could you elaborate on what you mean by “how it was set up”, please? Do you mean you want to see all of the installations which occurred, for example?

Yes like the setup_commands

Hmm, I’ll have to see. I’ll get back to you as soon as possible on this.

Thanks!

Nice to hear!

Wait are you bypassing your k8s Ray clusters now? 😛

Haha no.. only got to the config toml creation step i dont have access to the cloud to bring up or down the resources 🫠