astrai-inference-router

What This Skill Does

The Astrai Inference Router acts as an intelligent gateway for all your LLM operations within OpenClaw. By acting as a sophisticated middleware, it intercepts your LLM requests and dynamically routes them to the most cost-effective and performant provider available. Beyond mere routing, it provides a comprehensive suite of privacy and governance features, including automatic PII stripping, EU-only data residency enforcement, and granular budget controls. It uses Bayesian learning to analyze your specific task requirements—whether coding, research, or creative writing—to ensure you never overpay for performance you don't need.

Installation

To integrate the Astrai Inference Router into your workflow, execute the following command in your terminal: clawhub install openclaw/skills/skills/beee003/astrai-inference-router

Once installed, obtain your API key from as-trai.com. Configure your environment by setting the ASTRAI_API_KEY variable. You can further customize your experience by setting PRIVACY_MODE to your desired level (standard, enhanced, or max) and DAILY_BUDGET to keep your infrastructure costs under control.

Use Cases

• Enterprise Cost Optimization: Automatically route high-volume internal agent tasks to cheaper, high-performance models while keeping complex logic on premium models.
• Data-Sensitive Environments: Use 'max' privacy mode to ensure that zero PII leaves your infrastructure, enabling compliance with strict internal data policies.
• Global Compliance: Force all LLM traffic through EU servers for GDPR compliance without needing to manage individual provider configurations.
• Resiliency Planning: Prevent agent downtime by leveraging the auto-failover mechanism, which instantly switches providers if your primary LLM endpoint suffers an outage.

Example Prompts

1. "Route my upcoming technical documentation generation through the most cost-effective model that supports 128k context."
2. "Update my Astrai configuration to max privacy mode and ensure all future requests are strictly routed through EU infrastructure."
3. "Summarize my last 100 API calls and calculate the total savings compared to the standard OpenAI direct-billing rates."

Tips & Limitations

• Start with Enhanced: For most users, 'enhanced' mode provides the best balance of privacy and performance without limiting the utility of the LLM responses.
• Budget Monitoring: Use the DAILY_BUDGET variable to avoid unexpected spikes in usage. Remember that setting this to 0 removes the cap.
• Regional Latency: If you enforce REGION=eu, note that you may experience slight latency variations depending on your physical proximity to the designated data centers.
• Monitoring: Frequently check your analytics dashboard at as-trai.com to refine your routing preferences based on the real-time cost-saving data provided.