Deep Infra

Setup

On first use, read setup.md to align activation boundaries, reliability goals, and routing preferences before making configuration changes.

When to Use

Use this skill when the user wants to connect an OpenAI-compatible workflow to DeepInfra, choose open-source and frontier models by task type, set safe fallbacks, and control cost drift over time.

Architecture

Memory lives in ~/deep-infra/. See memory-template.md for structure.

~/deep-infra/
├── memory.md            # Active routing profile and constraints
├── providers.md         # Confirmed provider and auth choices
├── routing-rules.md     # Task -> model and fallback policy
├── incidents.md         # Outages, rate limits, and recovery notes
└── budgets.md           # Spend guardrails and optimization actions

Quick Reference

Use the smallest relevant file for the current task.

Core Rules

1. Start from Workload Classes, Not Model Hype

• Classify requests first: coding, analysis, extraction, summarization, or long-context synthesis.
• Map each class to a primary model and a fallback before changing any defaults.

2. Keep Authentication Explicit and Verifiable

• Use DEEPINFRA_API_KEY from the local environment, never pasted into logs or chat memory.
• Validate auth with a minimal request before applying routing changes.

3. Design Fallbacks for Failure Modes, Not Convenience

• Separate fallback reasons: rate limit, provider outage, latency spike, or output quality failure.
• Keep at least one fallback from a different model family for resilience.

4. Leverage Open-Source Model Diversity

• DeepInfra hosts models from many providers (DeepSeek, Moonshot, MiniMax, StepFun, NVIDIA, and more).
• Use model diversity to build resilient fallback chains across independent model families.

5. Enforce Cost Boundaries Before Throughput Tuning

• Set cost ceilings by task class and check expected token burn before broad rollout.
• Route low-stakes tasks to cheaper models and reserve premium models for high-impact tasks.

6. Change One Layer at a Time

• Modify either model selection, fallback policy, or budget limits in a single iteration.
• After each change, run a quick verification prompt set and record outcome.

7. Record Decisions for Repeatability

• Save the final routing policy, rationale, and known tradeoffs in memory.
• Reuse proven policies instead of repeatedly rebuilding from scratch.

Common Traps