mrscraper

What This Skill Does

The mrscraper skill provides a robust bridge between the OpenClaw agent and the MrScraper API. It enables users to perform complex data extraction tasks from web pages that are otherwise protected by anti-bot measures, captchas, or IP-based restrictions. By leveraging the unblocker service (featuring stealth browser emulation and global IP rotation), the agent can effectively mimic human behavior to fetch HTML content that is typically unreachable. Once the content is retrieved, the skill facilitates AI-powered parsing to transform raw web data into structured formats based on natural language instructions. This skill operates as an API-only interface, meaning it avoids the overhead of managing local browsers or complex scraping dependencies, instead delegating processing to MrScraper’s optimized cloud infrastructure.

Installation

Installation is straightforward via the OpenClaw ecosystem. You can add this skill to your workspace by executing the following command in your terminal:

clawhub install openclaw/skills/skills/ai-mrscraper/mrscraper

Once installed, ensure you have an MRSCRAPER_API_TOKEN from the MrScraper dashboard. You should store this token securely in your environment variables. The skill does not require additional local scripting or background services to function.

Use Cases

• Competitive Intelligence: Scraping pricing data or product catalogs from e-commerce sites that block standard crawlers.
• Lead Generation: Extracting contact details and company profiles from business directories.
• Content Aggregation: Collecting news or industry updates from sites protected by bot detection systems.
• Research & Analysis: Gathering historical market data or public records for analysis.

Example Prompts

1. "Use mrscraper to open https://example-store.com/products and extract the names, current prices, and stock status into a JSON format."
2. "Rerun the existing scraper configuration 'tech-news-feed' and give me the latest 10 items found."
3. "Fetch the detailed results for the most recent scrape run and summarize the key data points found for me."

Tips & Limitations

• Security: Never hardcode your API token in scripts or share it in logs. Always use environment variables.
• Authentication: Be aware that different endpoints require different auth methods; the unblocker uses query parameters, while the platform API uses header-based auth.
• Complexity: Natural language instructions work best when specific fields are requested. If the scraper returns unexpected results, consider providing more granular instructions or updating the scraper configuration in the dashboard.