Introducing Burla: The World's Simplest Cluster Compute Software
By Burla
Summary
Topics Covered
- Scale Python to 1000 CPUs Instantly
- Hyperparameter Tune ML Across 32 Parallel Jobs
- Auto-Installs Missing ML Packages on Demand
- Track All Jobs with Full Logs Dashboard
Full Transcript
Hi everyone, my name is Jake and today I'm going to give you a super quick demo of Burla, the world's simplest cluster compute software. The goal with Burla is
compute software. The goal with Burla is to make it completely trivial for anyone, even complete beginners, to scale Python over thousands of CPUs in the cloud using any hardware like GPUs
and any software like custom Docker containers. As you can see here, I'm in
containers. As you can see here, I'm in our cluster dashboard and I've got about a thousand CPUs worth of compute running. Here's how easy it is to run
running. Here's how easy it is to run some Python code on all of them.
Here I am in my IDE uh and I've imported remote parallel map which is the only function in the Bura Python package. All
I need to do to make it work is pass it some Python function and a list of arguments to call it on. Here I'm just passing it this simple function that sleeps for 1 second uh and then has this
print statement at the bottom. uh and
I'm going to call it on a list of numbers from 0 to 1,00. Uh and what this is going to do is it's going to call this function on every single argument in the list at the same time, each in a
separate container on its own CPU in that cluster that I showed you earlier.
Let's give it a run.
Cool. That was pretty fast. So because
this sleeps for 1 second and is running a thousand in parallel, it only takes about 1 second to finish the whole job.
Uh but that wasn't a very useful example. Uh so I've got another one here
example. Uh so I've got another one here where I am hyperparameter tuning a machine learning model in parallel uh using this cluster of computers that I started earlier. Uh now something that's
started earlier. Uh now something that's interesting is if I go to the settings tab here, you'll see that I'm running uh the Python 3.12 docker image in this cluster. That is the container that my
cluster. That is the container that my code runs inside. Uh, and that image does not come with scikitlearn or xg boost installed. But that's okay because
boost installed. But that's okay because burla will figure it out and install it really quickly automatically.
Here's how this example works very quickly if you're not familiar with hyperparameter tuning. That is just when
hyperparameter tuning. That is just when you train a bunch of machine learning models uh with different parameters to see which ones give you the best results. So here I call my function
results. So here I call my function train model on uh 32 different sets of parameters and I pass function CPU equals 80 which just means that this
train model function is going to get 80 CPUs assigned to it when I call it.
There we are installing that package.
Awesome. We've got some results and we're done. Now it's really easy to keep track of all the jobs you're running in Vera. If you just go to the jobs tab, you can see everything that I've been running here, including this last function, train model, uh, and all
of the logs associated with it. Hope
this all makes sense. Thanks so much for watching. Uh, if you guys want to try
watching. Uh, if you guys want to try out Burla, please send me an email. My
email is jakebla.dev.
We have a form on the front of our website for anyone who wants to try it.
We'll give you a managed instance with some free compute. Burla is open source and designed to be self-hosted. We
really want you to do that. So, we'll
give you some free compute at the beginning, uh, but you should move to the self-hosted version, and we're happy to help get hands-on, uh, and get that installed and working for you. Um, I
hope this all makes sense. Have a great day. Uh, yes.
day. Uh, yes.
Loading video analysis...