Prometheus

What This Skill Does

The Prometheus skill provides a comprehensive toolkit for managing, optimizing, and troubleshooting Prometheus monitoring environments. It encapsulates deep expertise in TSDB management, cardinality control, and query optimization, allowing the OpenClaw agent to act as a Site Reliability Engineer (SRE) assistant. From diagnosing high-cardinality label explosions to crafting efficient alerting rules, this skill ensures that your observability stack remains performant, accurate, and actionable.

Installation

To integrate this skill into your environment, run the following command: clawhub install openclaw/skills/skills/ivangdavila/prom

Use Cases

• Cardinality Management: Identify and prune high-cardinality labels (like UUIDs or request IDs) that threaten to overwhelm your Prometheus storage and memory usage.
• Alerting Engineering: Design robust alert rules that follow SRE best practices, including the implementation of the for clause to prevent flapping and the integration of mandatory runbook_url labels for efficient incident response.
• PromQL Optimization: Debug complex queries, identify dangerous label matching patterns using and/or operators, and optimize metric selection to avoid global scans.
• Infrastructure Auditing: Audit scrape configurations, identify Pushgateway misuse, and validate histogram versus summary bucket strategies for optimal SLO tracking.

Example Prompts

1. "I am seeing a steady increase in prometheus_tsdb_head_series. Can you help me identify which label sets are contributing to the cardinality explosion and suggest a relabeling rule to drop them?"
2. "Review my current alert rule for high latency. I have no for clause and I'm getting spammed with notifications. How should I rewrite this to be more reliable?"
3. "Explain the difference between rate() and increase() and tell me why my 30s scrape interval makes my rate(metric[1m]) query unreliable."

Tips & Limitations

• Cardinality: Always monitor prometheus_tsdb_head_series. If this metric exceeds 1 million, immediate action is required to avoid system degradation.
• Histograms: Ensure your histogram buckets are tailored to your specific service's latency profile. Standard default buckets are often inadequate for high-performance applications.
• Alerting Strategy: Always alert on user-facing symptoms rather than infrastructure causes (e.g., alert on high latency or error rates rather than high CPU usage).
• Limitations: This skill focuses on advisory and diagnostic tasks. It does not directly modify your server's configuration files on disk, but provides the exact syntax and patterns required to do so safely. Use promtool check rules to validate all configuration changes before applying them to production.