web-freedom-toolkit

What This Skill Does

The web-freedom-toolkit (v8.0.0) is a state-of-the-art orchestration engine designed for server-side web navigation in restricted or hardened environments. It provides a three-tiered abstraction layer that abstracts the complexity of bypassing anti-bot measures such as Cloudflare, Akamai, and perimeter firewalls. By intelligently routing requests through Scrapling (lightning-fast, self-healing fetches), curl_cffi (advanced TLS/JA4 fingerprinting), and DrissionPage (full Chromium-based visual rendering), the toolkit ensures high-fidelity, undetectable interaction with modern web targets.

Installation

To integrate this toolkit into your OpenClaw environment, execute the following command in your terminal: clawhub install openclaw/skills/skills/biogod2020/web-freedom-toolkit

Ensure that your VPS environment has the necessary Chromium headers installed to support the Tier 3 D-Mode functionality. Once installed, the freedom engine can be invoked via your standard OpenClaw execution path.

Use Cases

• Stealth Data Acquisition: Perform high-frequency web scraping on websites with aggressive anti-bot protections without triggering blocks or captcha challenges.
• Complex Task Automation: Execute deep workflows that require JavaScript execution, such as navigating multi-step authentication, visual element interaction, or dynamic content loading.
• Fingerprint-Sensitive Navigation: Access sites that validate client-side TLS handshake patterns and JA4 fingerprints, ensuring your agent session appears identical to a standard user browsing from a desktop machine.
• Resilient Web Monitoring: Automate monitoring of websites that frequently change their DOM structure using Scrapling's self-healing fetch capabilities.

Example Prompts

1. "Use the web-freedom-toolkit to scrape the product listings from target-site.com, ensuring you mimic a standard Chrome user profile to avoid detection."
2. "I need to automate a login process for a portal; please switch to Tier 3 D-Mode for this task to ensure all JavaScript buttons are rendered correctly."
3. "Run a speed-optimized crawl on the target documentation site using the stealth-first strategy, and provide a summary of the extracted findings."

Tips & Limitations

• Choose Tiers Wisely: Use Tier 1 or 2 for simple data extraction to save server resources and increase speed. Reserve Tier 3 (Chromium) for tasks that strictly require interaction with complex JS rendering.
• Resource Monitoring: Tier 3 operations consume significantly higher RAM and CPU. Monitor your VPS usage if running multiple concurrent D-Mode instances.
• Maintenance: As anti-bot technologies evolve, ensure you keep the freedom_engine.py script updated via ClawHub to maintain compatibility with the latest handshake impersonation protocols.