Scaling Big Data Interactive Workloads across Kubernetes Cluster
The Jupyter Notebook Stack has become the "de facto" platform used by data scientists to interactively work on big data problems. With the popularity of deep learning, there is also an increasing need for resources to make deep learning effective. In this session, we will discuss how we brought support for Kubernetes into Jupyter Enterprise Gateway and touch on some best practices on how to scale an interactive big data workloads across a Kubernets managed cluster.
Luciano Resende is an Open Source AI Platform Architect with IBM's CODAIT group. He's a highly performant technical leader that embraces challenges and complex problems to drive breakthrough innovations. Luciano's expertise is in open source, and enterprise-grade AI platform technologies with about 20 years of experience successfully designing, building and delivering complex software in fortune 500 companies and open source. He has a strong background in open source big data platforms such as Apache Spark, and data science building blocks such as the Jupyter Notebook Stack and Apache Toree Sc