vllm-deploy-docker
Deploy vLLM using Docker (pre-built images or build-from-source) with NVIDIA GPU support and run the OpenAI-compatible server.
awq-quantization
Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss. Use when deploying large models (7B-70B) on limited GPU memory, when you need faster inference than GPTQ with better accuracy preservation, or for instruction-tuned and multimodal models. MLSys 2024 Best Paper Award winner.
main2main
The main2main skill guides an AI agent to adapt the latest vLLM main branch code for vLLM Ascend project.
Io.Github.Qso Graph/Qsp Mcp
QSP — relay MCP tools to any OpenAI-compatible local LLM (llama.cpp, Ollama, vLLM)
kernelgen-flagos
Unified GPU kernel operator generation skill. Automatically detects the target repository type (FlagGems, vLLM, or general Python/Triton) and dispatches to the appropriate specialized sub-skill. Also includes a feedback submission sub-skill for bug reports. Use this skill when the user wants to generate a GPU kernel operator, create a Triton kernel, or says things like "generate an operator", "create a kernel for X", or "/kernelgen-flagos". This single skill replaces the need to install kernelgen-general, kernelgen-for-flaggems, kernelgen-for-vllm, and kernelgen-submit-feedback separately.
inference-server
Start and test the prime-rl inference server. Use when asked to run inference, start vLLM, test a model, or launch the inference server.