Skip to main content

Introducing RoleBasedGroup

· One min read
SGLang Team
SGLang Team
RBG Contributors

RoleBasedGroup (RBG) is a Kubernetes API for orchestrating distributed, stateful AI inference workloads.

Why RBG?​

Traditional Kubernetes primitives are ill-suited for LLM inference services that:

  • Run as multi-role topologies (gateway/router/prefill/decode)
  • Are performance-sensitive to GPU/network topology
  • Require atomic, cross-role operations

RBG treats an inference service as a role-based group, managing it as a single coordinated unit.

The SCOPE Framework​

RBG provides five core capabilities:

  • Stable - Topology-aware deterministic operations
  • Coordination - Cross-role policy engine
  • Orchestration - Role dependencies and startup sequences
  • Performance - Topology-aware placement
  • Extensible - Future-proof deployment abstraction

Getting Started​

Check out our Quick Start Guide to begin deploying LLM inference services with RBG.