Advancing AI – Spark for Machine Learning

🌟 Explore Spark on Databricks for Machine Learning! 🌟

Join us as we dive into the world of Spark on Databricks and unravel how it’s transforming machine learning and data science workflows. In this video, we’ll break down what Spark really is, uncover its top benefits, and discuss its pivotal role in the realm of MLOps.

🔍 We’ll also show you some real-world applications, demonstrate how Spark excels in parallelising tasks, and share tips on how you can harness its power in your projects.

👍 Like what you see? Hit the like button, subscribe for more insights, and drop a comment below to share how you’re using or planning to use Spark in your data adventures.

📰 Blog :

🚀 Dbdemos – Scale pandas API with Databricks runtime as backend
:

00:00 – Introduction to Advancing AL
00:54 – Introducing Sepi and Stuart from Databricks
01:54 – What is Spark and its role in Data Science and ML workflows.
03:43 – What are some Data Science projects that can benefit from using Spark
06:05 – Demo
08:54 – Leveraging Spark in data science workflows
09:50 – Benefits of Spark for productionisation
10:46 – Real world use cases and Spark
12:43 – Using MLflow to do parallelised model inference
16:19 – Spark and MLOps
17:32 – Spark vs Ray for parallelisation

Check out our other video series on MLOps:
🎥 Advancing AI – Beginners Guide to MLflow:
🎥 Advancing AI – Intro to MLOps Gym:
🎥 Advancing AI – Getting started with Feature Store:

About us:
Gavi:
Sepi:
Stuart:



source