model-audit

What This Skill Does

The model-audit skill is a sophisticated diagnostic tool designed to keep your OpenClaw agent stack performing at peak efficiency. It bridges the gap between your current LLM configuration and the rapidly evolving AI marketplace by tapping into live pricing data from OpenRouter. By analyzing your openclaw.json file, the script categorizes your active models into specialized buckets: reasoning, code, fast, cheap, and vision. It then cross-references this against the latest industry benchmarks to highlight discrepancies where you might be overpaying for performance or missing out on newer, faster, or more capable models that have recently entered the market. The tool calculates potential monthly savings and suggests actionable swaps, ensuring your token budget is always optimized for your specific use cases.

Installation

To integrate this utility, ensure you have the OpenClaw environment initialized. First, set your API credentials by exporting your key: export OPENROUTER_API_KEY='your-key-here'. Install the skill directly from the repository using the terminal command: clawhub install openclaw/skills/skills/aiwithabidi/model-audit. Once installed, you can trigger the audit script by navigating to your skill base directory and running python3 {baseDir}/scripts/model_audit.py. Ensure your openclaw.json is properly configured to allow the script to scan your active model list.

Use Cases

This skill is ideal for AI-driven businesses, software engineers, and automation power users. Use it to conduct monthly cost-benefit analyses of your AI infrastructure, evaluate if a new model release (such as a fresh Gemini or DeepSeek iteration) is ready to replace legacy models in your workflow, or perform deep-dive comparisons between two models to see if the latency/cost difference justifies the switch. It is particularly effective for teams managing high-volume token consumption where small per-token savings scale into significant monthly overhead reductions.

Example Prompts

1. "OpenClaw, please run a full model audit and show me how much I could save if I switch my reasoning models to the latest recommended alternatives."
2. "Perform a comparison between my current production model claude-3.5-sonnet and gpt-4o, and tell me which is more cost-effective for my code-generation tasks."
3. "Identify the best, cheapest model for simple vision tasks in my current stack and list any potential upgrades I should consider."

Tips & Limitations

To get the most accurate audit, ensure your openclaw.json is kept up to date. The script relies on live data from OpenRouter, so a stable internet connection is required during execution. While the audit provides excellent recommendations, always perform a trial run on a non-production workload before swapping models in critical, performance-sensitive infrastructure. Be aware that 'fast' and 'cheap' metrics are based on token costs, so evaluate your specific throughput requirements alongside the model's intelligence level.