stealth-browser

What This Skill Does

The Stealth Browser skill is designed to overcome advanced anti-bot measures employed by websites, enabling robust browser automation where traditional tools like Playwright and Selenium fail. It leverages specialized tools, Camoufox and curl_cffi, to bypass sophisticated defenses such as Cloudflare Turnstile, Datadome, and aggressive fingerprinting techniques. This skill is particularly useful for scraping data from heavily protected sites like Airbnb and Yelp, or when playwright-stealth is no longer effective. The skill operates within the pybox distrobox for isolated execution, ensuring a secure and consistent environment. Camoufox, a modified Firefox build, offers superior fingerprint evasion, while curl_cffi provides a lightweight API-only solution for sites that don't require JavaScript execution.

Installation

To install the Stealth Browser skill, use the following command within your OpenClaw environment:

clawhub install openclaw/skills/skills/kesslerio/kesslerio-stealth-browser

After installation, you will need to set up the necessary tools within the pybox distrobox. This is a one-time setup:

distrobox-enter pybox -- python3.14 -m pip install camoufox curl_cffi

Note that Camoufox is a Firefox fork and will download automatically on its first run, requiring approximately 700MB of disk space.

Use Cases

This skill is ideal for scenarios where standard web scraping methods are blocked:

• Bypassing Cloudflare and Datadome: If you encounter Cloudflare challenges, "checking your browser" pages, or Datadome security checks, the Stealth Browser skill can help.
• Scraping Protected Websites: Access and scrape data from sites like Airbnb, Yelp, and other platforms known for aggressive anti-bot measures.
• Replacing Outdated Stealth Solutions: When playwright-stealth is no longer effective against evolving anti-bot technologies.
• API-based Data Extraction: For sites with well-defined API endpoints, curl_cffi offers a fast and efficient method for data retrieval without browser overhead.

Example Prompts

• "Please scrape the apartment listings from this Airbnb URL using the stealth browser: https://www.airbnb.com/s/San-Francisco--CA/homes?adults=1"
• "Can you fetch the business reviews for the first 5 restaurants on this Yelp page? Use Camoufox: https://www.yelp.com/biz/starbucks-san-francisco-111"
• "I'm getting blocked by Cloudflare on https://example.com. Try to retrieve the page content using the stealth browser skill."

Tips & Limitations

• Proxy Requirement: To avoid immediate blocks on sensitive sites like Airbnb and Yelp, you must use residential or mobile proxies. Datacenter IPs (AWS, DigitalOcean) are insufficient. Refer to references/proxy-setup.md for detailed proxy configuration.
• Python Version: Always explicitly use python3.14 within the pybox distrobox, as other Python versions may have incompatible package installations.
• Camoufox Size: Be aware of the ~700MB download size for Camoufox.
• curl_cffi Limitations: The curl_cffi tool is for API endpoints only and does not execute JavaScript. It is not suitable for scraping dynamic web pages that rely heavily on client-side rendering.
• Behavioral Analysis: Some sites employ behavioral analysis beyond simple fingerprinting. Mimicking human browsing patterns (e.g., mouse movements, typing speed, navigation) might be necessary for sustained access, though this skill focuses on technical bypasses.