ROCm vLLM Deployment Skill

Production-ready automation for deploying vLLM inference services on AMD ROCm GPUs using Docker Compose.

Features

• Environment Auto-Check - Detects and repairs missing dependencies
• Model Parameter Detection - Auto-reads config.json for optimal settings
• VRAM Estimation - Calculates memory requirements before deployment
• Secure Token Handling - Never writes tokens to compose files
• Structured Output - All logs and test results saved per-model
• Deployment Reports - Human-readable summary for each deployment
• Health Verification - Automated health checks and functional tests
• Troubleshooting Guide - Common issues and solutions

Environment Prerequisites

Recommended (for production): Add to ~/.bash_profile:

# HuggingFace authentication token (required for gated models)
export HF_TOKEN="hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

# Model cache directory (optional)
export HF_HOME="$HOME/models"

# Apply changes
source ~/.bash_profile

Not required for testing: The skill will proceed without these set:

• HF_TOKEN: Optional — public models work without it; gated models fail at download with clear error
• HF_HOME: Optional — defaults to /root/.cache/huggingface/hub

Environment Variable Detection

Priority Order:

1. Explicit parameter (highest) — Provided in task/request (e.g., hf_token: "xxx")
2. Environment variable — Already set in shell or from parent process
3. ~/.bash_profile — Source to load variables
4. Default value (lowest) — HF_HOME defaults to /root/.cache/huggingface/hub

Philosophy: Fail fast for configuration errors, fail at download time for authentication errors.

Helper Scripts

Location: <skill-dir>/scripts/

check-env.sh

Validate and load environment variables before deployment.

Usage:

# Basic check (HF_TOKEN optional, HF_HOME optional with default)
./scripts/check-env.sh

# Strict mode (HF_HOME required, fails if not set)
./scripts/check-env.sh --strict

# Quiet mode (minimal output, for automation)
./scripts/check-env.sh --quiet

# Test with environment variables
HF_TOKEN="hf_xxx" HF_HOME="/models" ./scripts/check-env.sh

Exit Codes:

Note: This script is optional. You can also directly run source ~/.bash_profile.

generate-report.sh

Generate human-readable deployment report after successful deployment.

Usage:

./scripts/generate-report.sh <model-id> <container-name> <port> <status> [model-load-time] [memory-used]