KServe Essentials: Building a Production-Ready Cloud-Native Model Serving Platform
Abstract:
Despite significant advancements in ML/AI, creating large-scale data science and machine learning applications continues to be challenging. The variety of machine learning frameworks, hardware accelerators, cloud vendors as well as the complexity of data science workflows brings new challenges to MLOps. One particular challenge is that it’s non-trivial to build an inference system that’s suitable for a variety of models, including traditional ML models and large generative AI models. In this talk, we will explore how KServe addresses these challenges and supports the development of a production-ready cloud-native model serving platform. We would also like to invite all end users to share your use cases and help drive the project’s direction.