decodo-scraper

What This Skill Does

The decodo-scraper skill serves as a comprehensive interface for the Decodo Web Scraping API, enabling OpenClaw agents to perform advanced data gathering tasks. This skill provides an abstraction layer over complex web interactions, allowing users to execute Google searches, perform universal web page scraping, retrieve structured Amazon product data, extract YouTube video subtitles, and pull Reddit discussion content directly into their workflow. By converting web content into LLM-friendly formats like Markdown or parsed JSON, it drastically reduces token overhead while maintaining high data integrity.

Installation

To integrate this skill into your environment, use the OpenClaw command-line interface: clawhub install openclaw/skills/skills/donatasdecodo/decodo-scraper-skill

After installation, you must configure your authentication by setting the DECODO_AUTH_TOKEN environment variable. You can obtain this token from your Decodo Dashboard under the Scraping APIs section. It is recommended to store this in a .env file within your project root to ensure it remains secure and persistent across sessions.

Use Cases

This skill is indispensable for market research, content monitoring, and automated data collection. Use it to track Amazon product price trends, summarize long-form YouTube video content without watching them, aggregate Reddit sentiment on specific topics, or scrape documentation pages for real-time information retrieval. Because it handles the heavy lifting of proxy management and bot detection avoidance, it allows the agent to focus purely on analysis and synthesis.

Example Prompts

1. "Use the decodo-scraper to search Google for 'best noise-canceling headphones 2025 reviews' and summarize the top organic results for me."
2. "Extract the transcript from this YouTube URL [https://www.youtube.com/watch?v=...] using the decodo-scraper and write a 3-bullet point summary of the main arguments."
3. "Scrape the product details from this Amazon link [https://www.amazon.com/dp/...] and output the pricing and key features in a clean JSON format."

Tips & Limitations

• Token Optimization: Always prefer the universal target for general web pages, as it outputs Markdown, which is significantly more efficient for LLM context windows than raw HTML.
• Error Handling: The skill follows standard UNIX practices; if a request fails, it writes a JSON-formatted error to stderr and exits with status code 1. Ensure your agent logs are monitoring stderr to catch these failures.
• Rate Limiting: While the Decodo API manages proxy rotation, excessive requests in a short time frame may still encounter API limits. Adjust your request frequency accordingly.
• Authentication: Always verify your DECODO_AUTH_TOKEN if you receive a 401 or 403 error status code.