model-audit

What This Skill Does

The Model Audit skill is a comprehensive diagnostic tool for your OpenClaw agent, designed to keep your LLM infrastructure lean, performant, and cost-effective. By integrating directly with the OpenRouter API, it provides real-time oversight of your token spending and model performance. The skill continuously monitors your configured models (as defined in openclaw.json) and cross-references them against the latest available benchmarks, pricing, and capabilities on the market. It effectively transforms raw pricing data into actionable intelligence, categorizing your current stack into domains such as 'reasoning', 'code', 'fast', 'cheap', and 'vision'.

Installation

To integrate this skill into your environment, use the command-line interface provided by ClawHub:

clawhub install openclaw/skills/skills/aiwithabidi/model-audit-pro

Ensure that you have set the OPENROUTER_API_KEY in your system environment variables. Once installed, you can trigger the audit script directly from your terminal using python3 {baseDir}/scripts/model_audit.py.

Use Cases

• Cost Optimization: Automatically identify expensive models that can be swapped for lower-cost alternatives without compromising performance.
• Performance Benchmarking: Stay on the cutting edge by receiving suggestions to upgrade to newer, more efficient models as they are released.
• Stack Health Checks: Ensure your current model configuration provides sufficient redundancy across all required categories (code, vision, reasoning).
• Budget Forecasting: Analyze potential monthly savings based on your historical usage patterns compared to market rates.

Example Prompts

1. "Perform a full audit of my current LLM stack and show me where I can save money on my monthly bill."
2. "Compare my current models against the top-performing reasoning models and suggest an upgrade for complex tasks."
3. "Show me the best models for coding tasks under $0.50 per million tokens."

Tips & Limitations

• Tips: Run this audit once a month to capture changes in model pricing; many providers adjust rates frequently as newer versions are released. Use the --json flag if you are building an automated reporting dashboard to track your AI costs over time.
• Limitations: This skill relies strictly on the availability of data from the OpenRouter API. If the API is unreachable, the tool will be unable to generate recommendations. It assumes that your openclaw.json is accurately mapped to your actual agent usage patterns. The cost savings calculated are estimates based on standard token pricing and may vary based on your specific prompt/completion ratio.