We're excited to announce the release of RBG v0.7.0-alpha.2!

What's New​

This alpha release includes several improvements and new features. Please check out the release notes for more details.

Getting Started​

Install the latest version:

kubectl apply --server-side -f https://github.com/sgl-project/rbg/releases/download/v0.7.0-alpha.2/manifests.yaml

Feedback​

We welcome your feedback! Please report any issues on GitHub.

We're excited to announce the release of RBG v0.4.0!

Key Features​

This release includes:

• Improved multi-role coordination
• Enhanced autoscaling support
• Better failure handling mechanisms

Check out the release notes for complete details.

Version Compatibility​

RoleBasedGroup (RBG) is a Kubernetes API for orchestrating distributed, stateful AI inference workloads.

Why RBG?​

Traditional Kubernetes primitives are ill-suited for LLM inference services that:

• Run as multi-role topologies (gateway/router/prefill/decode)
• Are performance-sensitive to GPU/network topology
• Require atomic, cross-role operations

RBG treats an inference service as a role-based group, managing it as a single coordinated unit.

The SCOPE Framework​

RBG provides five core capabilities:

• Stable - Topology-aware deterministic operations
• Coordination - Cross-role policy engine
• Orchestration - Role dependencies and startup sequences
• Performance - Topology-aware placement
• Extensible - Future-proof deployment abstraction

Getting Started​

Check out our Quick Start Guide to begin deploying LLM inference services with RBG.