Blogs
292026
- Nov 2, 2026
- Sep 29, 2026
- Sep 23, 2026
- Sep 15, 2026
- Apr 16, 2026
- Apr 2, 2026
- Apr 1, 2026
- Mar 16, 2026
- Mar 9, 2026
2025
- Dec 24, 2025
- Nov 26, 2025
- Oct 31, 2025
- Sep 30, 2025
- Sep 23, 2025
- Sep 15, 2025
- Aug 8, 2025
- May 1, 2025
- Apr 30, 2025
- Feb 12, 2025
- Jan 30, 2025
2024
- Dec 3, 2024
- Nov 24, 2024
- Nov 1, 2024
- Sep 1, 2024
- Jul 2, 2024
- Jun 1, 2024
- Jan 16, 2024
2023
- Jan 1, 2023
2022
- Jun 1, 2022
Talks
9From Memory Budget to Cost Per Token: Serving Agentic Traffic with Large Open Weight Models
Yuchen Fama (Red Hat), Ashish Kamra (Red Hat)
GuideLLM v0.7.0+: Benchmarking Reasoning, Tool Calling, and Agentic Traces
Samuel Monson (Red Hat), Jared O'Connell (Red Hat)
KV Cache-Aware Routing and P/D Disaggregation on Kubernetes
Yuchen Fama & Ashish Kamra (Red Hat)
A Cross-Industry Benchmarking Tutorial for Distributed LLM Inference on Kubernetes
Samuel Monson (Red Hat), Ganesh Kudleppanavar (NVIDIA), Jason Kramberger (Google), Jing Chen (IBM Research)
Routing Stateful AI Workloads in Kubernetes
Maroon Ayoub (IBM Research), Michey Mehta (Red Hat)
Learn How to Run an LLM Inference Performance Benchmark on NVIDIA GPUs
Samuel Monson (Red Hat), Ashish Kamra (Red Hat)
Multi-Node Finetuning LLMs on Kubernetes: A Practitioner's Guide
Ashish Kamra (Red Hat), Boaz Ben Shabat (Red Hat)
Optimizing Load Balancing and Autoscaling for Large Language Model (LLM) Inference on Kubernetes
David Gray (Red Hat)
Efficiently Deploying and Benchmarking LLMs in Kubernetes
Nikhil Palaskar (Red Hat)
Publications
1llm-tuna: Hyperparameter Optimization for LLM Inference
An open-source framework that automates vLLM inference hyperparameter optimization using Bayesian search via Optuna, achieving up to 32.9% throughput improvement on mixture-of-experts models.
Featured Projects
8GuideLLM
SLO-aware benchmarking and evaluation platform for LLM deployments that simulates production workloads against OpenAI-compatible and vLLM-native servers.
MaintainerPerformance-Analysis-Agent
AI-powered analysis of vLLM benchmark data across models, versions, accelerators, and workloads.
Creatorvllm-profiler
Kubernetes mutating admission webhook that transparently injects PyTorch profiler instrumentation into vLLM serving pods, capturing CPU and CUDA traces with no code changes or rebuilds.
Creatorfournos
Kubernetes operator that schedules benchmark jobs via Kueue and executes them as Tekton PipelineRuns on remote clusters through the FORGE framework.
Creatorforge
Framework for Orchestrating Runtime Gen-AI Experiments — a CI-first harness for reproducible, observable performance and scale testing of AI/ML workloads on OpenShift.
Creatorinference-playbooks
Curated, reproducible deployment recipes that connect a model, serving stack, hardware profile, workload, and auditable benchmark evidence for running LLMs on OpenShift/Kubernetes.
Creatorserveit-studio
Automated LLM inference optimization for Kubernetes — deploys real vLLM instances, sweeps configurations across aggregated, disaggregated, and expert-parallel architectures, and ranks the optimal setup by TTFT and throughput.
Creatorself-hosted-llm-cost-calculator
Interactive cost calculator comparing self-hosted open-weight LLM inference against frontier API providers.
Creator