vllm配置 Prometheus + Grafana可视化监控步骤及看板无数据问题纠正
·
vLLM 原生内置了 Prometheus 格式的 /metrics 接口，配合 Prometheus + Grafana 即可实现指标可视化。以下是 Docker 一键启动的完整方案：
架构概览
vLLM (:8000/metrics) ──► Prometheus (:9090) ──► Grafana (:3000)
第一步：确认 vLLM 已暴露指标
vLLM 默认在 API 端口（如 8000）自动暴露 /metrics，无需额外参数 ：
curl http://localhost:8000/metrics | head -20
若看到 vllm:num_requests_running 等输出，说明指标正常暴露。
注意：部分旧版本或自定义部署可能需要显式指定
--metrics-port，但 vLLM 0.4+ 通常默认集成在 API 端口。
第二步：Docker Compose 一键启动监控栈
创建 docker-compose.yml：
version: '3.8'
services:
prometheus:
image: prom/prometheus:latest
container_name: prometheus
ports:
- "9090:9090"
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml
- prometheus_data:/prometheus
command:
- '--config.file=/etc/prometheus/prometheus.yml'
- '--storage.tsdb.path=/prometheus'
- '--storage.tsdb.retention.time=30d'
- '--web.enable-lifecycle'
restart: unless-stopped
grafana:
image: grafana/grafana:latest
container_name: grafana
ports:
- "3000:3000"
volumes:
- grafana_data:/var/lib/grafana
environment:
- GF_SECURITY_ADMIN_USER=admin
- GF_SECURITY_ADMIN_PASSWORD=admin123
restart: unless-stopped
volumes:
prometheus_data:
grafana_data:
创建 prometheus.yml（注意替换 IP 为你的 vLLM 宿主机 IP）：
global:
scrape_interval: 5s
scrape_configs:
- job_name: 'vllm'
static_configs:
- targets: ['host.docker.internal:8000'] # Mac/Win
# Linux 用户改为 vLLM 实际 IP，如 ['192.168.1.100:8000']
metrics_path: '/metrics'
scrape_interval: 5s
启动：
docker compose up -d
第三步：配置 Grafana
-
访问：
http://你的服务器IP:3000 -
登录：
admin/admin123 -
添加数据源：
- Configuration → Data Sources → Add data source → 选 Prometheus
- URL 填
http://prometheus:9090（容器内 DNS 名） - Save & Test
-
导入看板（推荐社区预制模板）：
- Dashboards → New → Import
- 输入看板 ID：
18602（社区 vLLM 专用看板） - 选择刚才添加的 Prometheus 数据源 → Import
核心监控指标说明
| 指标名 | 类型 | 含义 |
|---|---|---|
vllm:num_requests_running | Gauge | 当前正在 GPU 上执行的请求数 |
vllm:num_requests_waiting | Gauge | 等待队列长度 |
vllm:kv_cache_usage_perc | Gauge | KV Cache 使用率 (0-1) |
vllm:time_to_first_token_seconds | Histogram | 首 Token 延迟 (TTFT) |
vllm:e2e_request_latency_seconds | Histogram | 端到端请求延迟 |
vllm:generation_tokens_per_second | Gauge | Token 生成速度 |
完整指标列表参考 vLLM 官方文档 。
常见问题
- 容器内访问不到宿主机 vLLM：Linux 下
host.docker.internal可能不生效，需改为宿主机实际 IP 或--network host模式 - 指标名带冒号：vLLM 指标使用
vllm:前缀（如vllm:num_requests_running），Grafana 中查询时直接用完整名称即可 - GPU 硬件级监控：如需显存温度/功耗等硬件指标，可额外部署
nvcr.io/nvidia/k8s/dcgm-exporter:3.3.5并加入 Prometheus 抓取
“看板没数据”80% 是链路里某一段没通。按顺序排一下，很快能定位👇
🧠 一条完整链路长这样
vLLM → /metrics → Prometheus → Grafana → Dashboard
👉 任意一段断了 = 没数据
🔍 第一步：确认 vLLM 有没有暴露 metrics（最关键）
你现在的启动命令里 没有这个参数：
--metrics-port 8000
（新版vllm不用加，自动有！ 测一下有没有数据就可以！）
👉 没开的话，Prometheus 根本抓不到数据
✅ 验证（非常重要）
浏览器打开：
http://你的IP:8003/metrics
如果正常，你会看到一堆：
vllm_prompt_tokens_total
vllm_generation_tokens_total
❗如果这里都没有
👉 Grafana 一定不会有数据
🔍 第二步：检查 Prometheus 有没有抓到
打开 Prometheus：
http://你的IP:9090/targets
看你的 vLLM：
job: vllm
status: UP ✅
❌ 如果是 DOWN
说明：
- IP/端口写错
- 容器网络不通
- metrics 没开
✅ Prometheus 配置示例
scrape_configs:
- job_name: 'vllm'
static_configs:
- targets: ['你的IP:8003']
🔍 第三步：Prometheus 里查有没有数据
在 Prometheus UI 里试：
vllm_num_requests_running
如果：
- 有曲线 → OK
- 没数据 → 抓取失败
🔍 第四步：Grafana 数据源检查
在 Grafana：
- Settings → Data Sources
- 选 Prometheus
- 点 “Test”
👉 必须是：
Data source is working ✅
🔍 第五步：Dashboard 本身问题
常见坑👇
❌ 1. metric 名不匹配
有些模板用的是：
tgi_*
triton_*
👉 vLLM 是：
vllm_*
❌ 2. 时间范围不对
右上角：
Last 5 minutes / Last 1 hour
👉 刚启动时可能没数据
❌ 3. 没有请求流量
👉 vLLM metrics 是“实时累加”的
如果你没打请求：
👉 图是空的
🧪 最快自检方法（强烈推荐）
按顺序做👇
✅ 1. 打开 metrics 页面
http://IP:8003/metrics
✅ 2. 手动发一个请求
curl http://IP:8002/v1/chat/completions ...
✅ 3. 刷新 metrics
看：
vllm_generation_tokens_total
显示成功
官方看板配置-中文版
{
"annotations": [
{
"kind": "AnnotationQuery",
"spec": {
"builtIn": true,
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"query": {
"datasource": {
"name": "-- Grafana --"
},
"group": "grafana",
"kind": "DataQuery",
"spec": {
"limit": 100,
"matchAny": false,
"tags": [],
"type": "dashboard"
},
"version": "v0"
}
}
}
],
"cursorSync": "Off",
"description": "vLLM 推理服务器性能监控看板",
"editable": true,
"elements": {
"panel-10": {
"kind": "Panel",
"spec": {
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "builder",
"expr": "histogram_quantile(0.99, sum by(le) (rate(vllm:inter_token_latency_seconds_bucket{model_name=\"$model_name\"}[$__rate_interval])))",
"fullMetaSearch": false,
"includeNullMetadata": false,
"instant": false,
"legendFormat": "P99",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "A"
}
},
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "builder",
"expr": "histogram_quantile(0.95, sum by(le) (rate(vllm:inter_token_latency_seconds_bucket{model_name=\"$model_name\"}[$__rate_interval])))",
"fullMetaSearch": false,
"includeNullMetadata": false,
"instant": false,
"legendFormat": "P95",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "B"
}
},
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "builder",
"expr": "histogram_quantile(0.9, sum by(le) (rate(vllm:inter_token_latency_seconds_bucket{model_name=\"$model_name\"}[$__rate_interval])))",
"fullMetaSearch": false,
"includeNullMetadata": false,
"instant": false,
"legendFormat": "P90",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "C"
}
},
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "builder",
"expr": "histogram_quantile(0.5, sum by(le) (rate(vllm:inter_token_latency_seconds_bucket{model_name=\"$model_name\"}[$__rate_interval])))",
"fullMetaSearch": false,
"includeNullMetadata": false,
"instant": false,
"legendFormat": "P50",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "D"
}
},
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"editorMode": "code",
"expr": "rate(vllm:inter_token_latency_seconds_sum{model_name=\"$model_name\"}[$__rate_interval])\n/\nrate(vllm:inter_token_latency_seconds_count{model_name=\"$model_name\"}[$__rate_interval])",
"instant": false,
"legendFormat": "平均值",
"range": true
},
"version": "v0"
},
"refId": "E"
}
}
],
"queryOptions": {},
"transformations": []
}
},
"description": "Token之间的生成延迟，单位为秒",
"id": 10,
"links": [],
"title": "Token间延迟",
"vizConfig": {
"group": "timeseries",
"kind": "VizConfig",
"spec": {
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
},
{
"color": "red",
"value": 80
}
]
},
"unit": "s"
},
"overrides": []
},
"options": {
"annotations": {
"clustering": -1,
"multiLane": false
},
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"hideZeros": false,
"mode": "single",
"sort": "none"
}
}
},
"version": "13.0.1"
}
}
},
"panel-11": {
"kind": "Panel",
"spec": {
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "builder",
"expr": "sum by(finished_reason) (increase(vllm:request_success_total{model_name=\"$model_name\"}[$__rate_interval]))",
"fullMetaSearch": false,
"includeNullMetadata": true,
"instant": false,
"interval": "",
"legendFormat": "{{finished_reason}}",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "A"
}
}
],
"queryOptions": {},
"transformations": []
}
},
"description": "按完成原因统计的已结束请求数：生成 EOS 结束符或达到最大序列长度",
"id": 11,
"links": [],
"title": "完成原因",
"vizConfig": {
"group": "timeseries",
"kind": "VizConfig",
"spec": {
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
},
{
"color": "red",
"value": 80
}
]
}
},
"overrides": []
},
"options": {
"annotations": {
"clustering": -1,
"multiLane": false
},
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"hideZeros": false,
"mode": "single",
"sort": "none"
}
}
},
"version": "13.0.1"
}
}
},
"panel-12": {
"kind": "Panel",
"spec": {
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "builder",
"expr": "sum by(le) (increase(vllm:request_prompt_tokens_bucket{model_name=\"$model_name\"}[$__rate_interval]))",
"format": "heatmap",
"fullMetaSearch": false,
"includeNullMetadata": true,
"instant": false,
"legendFormat": "{{le}}",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "A"
}
}
],
"queryOptions": {},
"transformations": []
}
},
"description": "请求提示词长度的热力图分布",
"id": 12,
"links": [],
"title": "请求提示长度",
"vizConfig": {
"group": "heatmap",
"kind": "VizConfig",
"spec": {
"fieldConfig": {
"defaults": {
"custom": {
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"scaleDistribution": {
"type": "linear"
}
}
},
"overrides": []
},
"options": {
"annotations": {
"clustering": -1,
"multiLane": false
},
"calculate": false,
"cellGap": 1,
"cellValues": {
"unit": "none"
},
"color": {
"exponent": 0.5,
"fill": "dark-orange",
"min": 0,
"mode": "scheme",
"reverse": false,
"scale": "exponential",
"scheme": "Spectral",
"steps": 64
},
"exemplars": {
"color": "rgba(255,0,255,0.7)"
},
"filterValues": {
"le": 1e-09
},
"legend": {
"show": true
},
"rowsFrame": {
"layout": "auto",
"value": "请求数量"
},
"tooltip": {
"mode": "single",
"showColorScale": false,
"yHistogram": true
},
"yAxis": {
"axisLabel": "提示词长度",
"axisPlacement": "left",
"reverse": false,
"unit": "none"
}
}
},
"version": "13.0.1"
}
}
},
"panel-13": {
"kind": "Panel",
"spec": {
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "builder",
"expr": "sum by(le) (increase(vllm:request_generation_tokens_bucket{model_name=\"$model_name\"}[$__rate_interval]))",
"format": "heatmap",
"fullMetaSearch": false,
"includeNullMetadata": true,
"instant": false,
"legendFormat": "{{le}}",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "A"
}
}
],
"queryOptions": {},
"transformations": []
}
},
"description": "请求生成内容长度的热力图分布",
"id": 13,
"links": [],
"title": "请求生成长度",
"vizConfig": {
"group": "heatmap",
"kind": "VizConfig",
"spec": {
"fieldConfig": {
"defaults": {
"custom": {
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"scaleDistribution": {
"type": "linear"
}
}
},
"overrides": []
},
"options": {
"annotations": {
"clustering": -1,
"multiLane": false
},
"calculate": false,
"cellGap": 1,
"cellValues": {
"unit": "none"
},
"color": {
"exponent": 0.5,
"fill": "dark-orange",
"min": 0,
"mode": "scheme",
"reverse": false,
"scale": "exponential",
"scheme": "Spectral",
"steps": 64
},
"exemplars": {
"color": "rgba(255,0,255,0.7)"
},
"filterValues": {
"le": 1e-09
},
"legend": {
"show": true
},
"rowsFrame": {
"layout": "auto",
"value": "请求数量"
},
"tooltip": {
"mode": "single",
"showColorScale": false,
"yHistogram": true
},
"yAxis": {
"axisLabel": "生成长度",
"axisPlacement": "left",
"reverse": false,
"unit": "none"
}
}
},
"version": "13.0.1"
}
}
},
"panel-14": {
"kind": "Panel",
"spec": {
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "code",
"expr": "rate(vllm:request_queue_time_seconds_sum{model_name=\"$model_name\"}[$__rate_interval])",
"fullMetaSearch": false,
"includeNullMetadata": true,
"instant": false,
"legendFormat": "{{finished_reason}}",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "A"
}
}
],
"queryOptions": {},
"transformations": []
}
},
"description": "请求在队列中的等待时间",
"id": 14,
"links": [],
"title": "队列等待时间",
"vizConfig": {
"group": "timeseries",
"kind": "VizConfig",
"spec": {
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "秒",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
},
{
"color": "red",
"value": 80
}
]
}
},
"overrides": []
},
"options": {
"annotations": {
"clustering": -1,
"multiLane": false
},
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"hideZeros": false,
"mode": "single",
"sort": "none"
}
}
},
"version": "13.0.1"
}
}
},
"panel-15": {
"kind": "Panel",
"spec": {
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "code",
"expr": "rate(vllm:request_prefill_time_seconds_sum{model_name=\"$model_name\"}[$__rate_interval])",
"fullMetaSearch": false,
"includeNullMetadata": true,
"instant": false,
"legendFormat": "预填充",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "A"
}
},
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"editorMode": "code",
"expr": "rate(vllm:request_decode_time_seconds_sum{model_name=\"$model_name\"}[$__rate_interval])",
"instant": false,
"legendFormat": "解码",
"range": true
},
"version": "v0"
},
"refId": "B"
}
}
],
"queryOptions": {},
"transformations": []
}
},
"description": "请求的预填充（Prefill）和解码（Decode）阶段耗时",
"id": 15,
"links": [],
"title": "预填充与解码时间",
"vizConfig": {
"group": "timeseries",
"kind": "VizConfig",
"spec": {
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
},
{
"color": "red",
"value": 80
}
]
}
},
"overrides": []
},
"options": {
"annotations": {
"clustering": -1,
"multiLane": false
},
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"hideZeros": false,
"mode": "single",
"sort": "none"
}
}
},
"version": "13.0.1"
}
}
},
"panel-16": {
"kind": "Panel",
"spec": {
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "code",
"expr": "rate(vllm:request_max_num_generation_tokens_sum{model_name=\"$model_name\"}[$__rate_interval])",
"fullMetaSearch": false,
"includeNullMetadata": true,
"instant": false,
"legendFormat": "Token数",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "A"
}
}
],
"queryOptions": {},
"transformations": []
}
},
"description": "序列组中允许生成的最大Token数量",
"id": 16,
"links": [],
"title": "序列组最大生成Token数",
"vizConfig": {
"group": "timeseries",
"kind": "VizConfig",
"spec": {
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
},
{
"color": "red",
"value": 80
}
]
}
},
"overrides": []
},
"options": {
"annotations": {
"clustering": -1,
"multiLane": false
},
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"hideZeros": false,
"mode": "single",
"sort": "none"
}
}
},
"version": "13.0.1"
}
}
},
"panel-3": {
"kind": "Panel",
"spec": {
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "builder",
"expr": "vllm:num_requests_running{model_name=\"$model_name\"}",
"fullMetaSearch": false,
"includeNullMetadata": true,
"instant": false,
"legendFormat": "运行中数量",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "A"
}
},
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "builder",
"expr": "vllm:num_requests_waiting{model_name=\"$model_name\"}",
"fullMetaSearch": false,
"includeNullMetadata": true,
"instant": false,
"legendFormat": "等待中数量",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "C"
}
}
],
"queryOptions": {},
"transformations": []
}
},
"description": "RUNNING（运行中）、WAITING（等待中）和 SWAPPED（已交换）状态的请求数量",
"id": 3,
"links": [],
"title": "调度器状态",
"vizConfig": {
"group": "timeseries",
"kind": "VizConfig",
"spec": {
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
},
{
"color": "red",
"value": 80
}
]
},
"unit": "none"
},
"overrides": []
},
"options": {
"annotations": {
"clustering": -1,
"multiLane": false
},
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"hideZeros": false,
"mode": "single",
"sort": "none"
}
}
},
"version": "13.0.1"
}
}
},
"panel-4": {
"kind": "Panel",
"spec": {
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"editorMode": "code",
"expr": "vllm:kv_cache_usage_perc{model_name=\"$model_name\"}",
"instant": false,
"legendFormat": "GPU缓存使用率",
"range": true
},
"version": "v0"
},
"refId": "A"
}
}
],
"queryOptions": {},
"transformations": []
}
},
"description": "vLLM 已使用的缓存块百分比",
"id": 4,
"links": [],
"title": "缓存利用率",
"vizConfig": {
"group": "timeseries",
"kind": "VizConfig",
"spec": {
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
},
{
"color": "red",
"value": 80
}
]
},
"unit": "percentunit"
},
"overrides": []
},
"options": {
"annotations": {
"clustering": -1,
"multiLane": false
},
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"hideZeros": false,
"mode": "single",
"sort": "none"
}
}
},
"version": "13.0.1"
}
}
},
"panel-5": {
"kind": "Panel",
"spec": {
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "builder",
"expr": "histogram_quantile(0.99, sum by(le) (rate(vllm:time_to_first_token_seconds_bucket{model_name=\"$model_name\"}[$__rate_interval])))",
"fullMetaSearch": false,
"includeNullMetadata": false,
"instant": false,
"legendFormat": "P99",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "A"
}
},
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "builder",
"expr": "histogram_quantile(0.95, sum by(le) (rate(vllm:time_to_first_token_seconds_bucket{model_name=\"$model_name\"}[$__rate_interval])))",
"fullMetaSearch": false,
"includeNullMetadata": false,
"instant": false,
"legendFormat": "P95",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "B"
}
},
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "builder",
"expr": "histogram_quantile(0.9, sum by(le) (rate(vllm:time_to_first_token_seconds_bucket{model_name=\"$model_name\"}[$__rate_interval])))",
"fullMetaSearch": false,
"includeNullMetadata": false,
"instant": false,
"legendFormat": "P90",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "C"
}
},
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "builder",
"expr": "histogram_quantile(0.5, sum by(le) (rate(vllm:time_to_first_token_seconds_bucket{model_name=\"$model_name\"}[$__rate_interval])))",
"fullMetaSearch": false,
"includeNullMetadata": false,
"instant": false,
"legendFormat": "P50",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "D"
}
},
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"editorMode": "code",
"expr": "rate(vllm:time_to_first_token_seconds_sum{model_name=\"$model_name\"}[$__rate_interval])\n/\nrate(vllm:time_to_first_token_seconds_count{model_name=\"$model_name\"}[$__rate_interval])",
"instant": false,
"legendFormat": "平均值",
"range": true
},
"version": "v0"
},
"refId": "E"
}
}
],
"queryOptions": {},
"transformations": []
}
},
"description": "P50、P90、P95、P99 的首Token延迟，单位为秒",
"id": 5,
"links": [],
"title": "首Token延迟 (TTFT)",
"vizConfig": {
"group": "timeseries",
"kind": "VizConfig",
"spec": {
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
},
{
"color": "red",
"value": 80
}
]
},
"unit": "s"
},
"overrides": []
},
"options": {
"annotations": {
"clustering": -1,
"multiLane": false
},
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"hideZeros": false,
"mode": "single",
"sort": "none"
}
}
},
"version": "13.0.1"
}
}
},
"panel-8": {
"kind": "Panel",
"spec": {
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "builder",
"expr": "rate(vllm:prompt_tokens_total{model_name=\"$model_name\"}[$__rate_interval])",
"fullMetaSearch": false,
"includeNullMetadata": false,
"instant": false,
"legendFormat": "提示Token/秒",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "A"
}
},
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "builder",
"expr": "rate(vllm:generation_tokens_total{model_name=\"$model_name\"}[$__rate_interval])",
"fullMetaSearch": false,
"includeNullMetadata": false,
"instant": false,
"legendFormat": "生成Token/秒",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "B"
}
}
],
"queryOptions": {},
"transformations": []
}
},
"description": "每秒处理的Token数量",
"id": 8,
"links": [],
"title": "Token吞吐量",
"vizConfig": {
"group": "timeseries",
"kind": "VizConfig",
"spec": {
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
},
{
"color": "red",
"value": 80
}
]
}
},
"overrides": []
},
"options": {
"annotations": {
"clustering": -1,
"multiLane": false
},
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"hideZeros": false,
"mode": "single",
"sort": "none"
}
}
},
"version": "13.0.1"
}
}
},
"panel-9": {
"kind": "Panel",
"spec": {
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "builder",
"expr": "histogram_quantile(0.99, sum by(le) (rate(vllm:e2e_request_latency_seconds_bucket{model_name=\"$model_name\"}[$__rate_interval])))",
"fullMetaSearch": false,
"includeNullMetadata": false,
"instant": false,
"legendFormat": "P99",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "A"
}
},
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "builder",
"expr": "histogram_quantile(0.95, sum by(le) (rate(vllm:e2e_request_latency_seconds_bucket{model_name=\"$model_name\"}[$__rate_interval])))",
"fullMetaSearch": false,
"includeNullMetadata": false,
"instant": false,
"legendFormat": "P95",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "B"
}
},
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "builder",
"expr": "histogram_quantile(0.9, sum by(le) (rate(vllm:e2e_request_latency_seconds_bucket{model_name=\"$model_name\"}[$__rate_interval])))",
"fullMetaSearch": false,
"includeNullMetadata": false,
"instant": false,
"legendFormat": "P90",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "C"
}
},
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"disableTextWrap": false,
"editorMode": "builder",
"expr": "histogram_quantile(0.5, sum by(le) (rate(vllm:e2e_request_latency_seconds_bucket{model_name=\"$model_name\"}[$__rate_interval])))",
"fullMetaSearch": false,
"includeNullMetadata": false,
"instant": false,
"legendFormat": "P50",
"range": true,
"useBackend": false
},
"version": "v0"
},
"refId": "D"
}
},
{
"kind": "PanelQuery",
"spec": {
"hidden": false,
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"editorMode": "code",
"expr": "rate(vllm:e2e_request_latency_seconds_sum{model_name=\"$model_name\"}[$__rate_interval])\n/\nrate(vllm:e2e_request_latency_seconds_count{model_name=\"$model_name\"}[$__rate_interval])",
"instant": false,
"legendFormat": "平均值",
"range": true
},
"version": "v0"
},
"refId": "E"
}
}
],
"queryOptions": {},
"transformations": []
}
},
"description": "端到端请求延迟，单位为秒",
"id": 9,
"links": [],
"title": "端到端请求延迟",
"vizConfig": {
"group": "timeseries",
"kind": "VizConfig",
"spec": {
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"showValues": false,
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
},
{
"color": "red",
"value": 80
}
]
},
"unit": "s"
},
"overrides": []
},
"options": {
"annotations": {
"clustering": -1,
"multiLane": false
},
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"hideZeros": false,
"mode": "single",
"sort": "none"
}
}
},
"version": "13.0.1"
}
}
}
},
"layout": {
"kind": "GridLayout",
"spec": {
"items": [
{
"kind": "GridLayoutItem",
"spec": {
"element": {
"kind": "ElementReference",
"name": "panel-9"
},
"height": 8,
"width": 12,
"x": 0,
"y": 0
}
},
{
"kind": "GridLayoutItem",
"spec": {
"element": {
"kind": "ElementReference",
"name": "panel-8"
},
"height": 8,
"width": 12,
"x": 12,
"y": 0
}
},
{
"kind": "GridLayoutItem",
"spec": {
"element": {
"kind": "ElementReference",
"name": "panel-10"
},
"height": 8,
"width": 12,
"x": 0,
"y": 8
}
},
{
"kind": "GridLayoutItem",
"spec": {
"element": {
"kind": "ElementReference",
"name": "panel-3"
},
"height": 8,
"width": 12,
"x": 12,
"y": 8
}
},
{
"kind": "GridLayoutItem",
"spec": {
"element": {
"kind": "ElementReference",
"name": "panel-5"
},
"height": 8,
"width": 12,
"x": 0,
"y": 16
}
},
{
"kind": "GridLayoutItem",
"spec": {
"element": {
"kind": "ElementReference",
"name": "panel-4"
},
"height": 8,
"width": 12,
"x": 12,
"y": 16
}
},
{
"kind": "GridLayoutItem",
"spec": {
"element": {
"kind": "ElementReference",
"name": "panel-12"
},
"height": 8,
"width": 12,
"x": 0,
"y": 24
}
},
{
"kind": "GridLayoutItem",
"spec": {
"element": {
"kind": "ElementReference",
"name": "panel-13"
},
"height": 8,
"width": 12,
"x": 12,
"y": 24
}
},
{
"kind": "GridLayoutItem",
"spec": {
"element": {
"kind": "ElementReference",
"name": "panel-11"
},
"height": 8,
"width": 12,
"x": 0,
"y": 32
}
},
{
"kind": "GridLayoutItem",
"spec": {
"element": {
"kind": "ElementReference",
"name": "panel-14"
},
"height": 8,
"width": 12,
"x": 12,
"y": 32
}
},
{
"kind": "GridLayoutItem",
"spec": {
"element": {
"kind": "ElementReference",
"name": "panel-15"
},
"height": 8,
"width": 12,
"x": 0,
"y": 40
}
},
{
"kind": "GridLayoutItem",
"spec": {
"element": {
"kind": "ElementReference",
"name": "panel-16"
},
"height": 8,
"width": 12,
"x": 12,
"y": 40
}
}
]
}
},
"links": [],
"liveNow": false,
"preload": false,
"tags": [],
"timeSettings": {
"autoRefresh": "",
"autoRefreshIntervals": [
"5s",
"10s",
"30s",
"1m",
"5m",
"15m",
"30m",
"1h",
"2h",
"1d"
],
"fiscalYearStartMonth": 0,
"from": "now-6h",
"hideTimepicker": false,
"timezone": "browser",
"to": "now"
},
"title": "vLLM 推理服务监控",
"variables": [
{
"kind": "DatasourceVariable",
"spec": {
"allowCustomValue": true,
"current": {
"text": "prometheus",
"value": "ffl8d5fs05j40a"
},
"hide": "dontHide",
"includeAll": false,
"label": "数据源",
"multi": false,
"name": "DS_PROMETHEUS",
"options": [],
"pluginId": "prometheus",
"refresh": "onDashboardLoad",
"regex": "",
"skipUrlSync": false
}
},
{
"kind": "QueryVariable",
"spec": {
"allowCustomValue": true,
"current": {
"text": "/model",
"value": "/model"
},
"definition": "label_values(model_name)",
"hide": "dontHide",
"includeAll": false,
"label": "模型名称",
"multi": false,
"name": "model_name",
"options": [],
"query": {
"datasource": {
"name": "${DS_PROMETHEUS}"
},
"group": "prometheus",
"kind": "DataQuery",
"spec": {
"query": "label_values(model_name)",
"refId": "StandardVariableQuery"
},
"version": "v0"
},
"refresh": "onDashboardLoad",
"regex": "",
"regexApplyTo": "value",
"skipUrlSync": false,
"sort": "disabled"
}
}
]
}
DAMO开发者矩阵，由阿里巴巴达摩院和中国互联网协会联合发起，致力于探讨最前沿的技术趋势与应用成果，搭建高质量的交流与分享平台，推动技术创新与产业应用链接，围绕“人工智能与新型计算”构建开放共享的开发者生态。
更多推荐
7
0- 0
已为社区贡献5条内容
所有评论(0)