KServe
Model serving framework for deploying and managing inference services.
Component Category
AI serving / model inference
Component Description
KServe is a Kubernetes-native model serving framework that provides a standardized interface for deploying, scaling, and managing machine learning models. It supports multiple serving runtimes and defines the InferenceService custom resource for declarative model deployment. The platform deploys both the KServe CRDs (sync wave 6) and the KServe controller (sync wave 7) as separate components.
Why It Is Used
In BullSequana AI, KServe provides the target model serving framework for the platform. It offers a standardized, framework-agnostic serving layer with support for custom pre/post-processing, model explainability, and multi-model serving patterns through the InferenceService API. KServe is planned to fully replace KubeAI as the primary inference orchestration layer.
Learn More
Deployment notes
KServe is split into two platform components: kserve-crd (CRDs only, sync wave 6) and kserve (controller, sync wave 7). The CRDs must be installed before the controller starts. Both deploy into the kserve namespace. The controller creates additional resources for Rook Ceph RGW integration for model storage.
Interacts With
KubeAI, which KServe is planned to fully replace as the inference orchestration layer.Rook Ceph, which provides S3-compatible storage for model artifacts through RGW.Gateway API, which routes inference traffic to KServe endpoints.