When you think of 'Spark', Matei Zaharia should come to mind instantly!
He laid the groundwork for Spark in the community, and together with Alexy Khrabrov, they launched the world's first Spark meetup. 🏆
Let's say that Spark is in our DNA here in the Communities By the Bay 🧬
Today's a great day to revisit Matei Zaharia's insightful talk at SBTB 2020.
In his talk Matei Zaharia discussed how they built Databricks' large-scale cloud service, managing millions of VMs and exabytes of data using Scala, Kubernetes, Envoy, and Prometheus.
Explored the design patterns and engineering processes they've adopted, and the changes were made to improve data analytics systems in the cloud, including ACID storage system and Apache Spark's auto-scaling and auto-shutdown features.
Please, enjoy the whole video:
It's full of valuable insights and may inspire you for something great!
Meanwhile, join us in November as we continue the adventure of Spark at SBTB 2023! ❤️