web-multi-search

What This Skill Does

The web-multi-search skill for OpenClaw provides a powerful, aggregated searching capability by leveraging the async-search-scraper library. Instead of relying on a single source of truth, this agent skill queries multiple search engines—specifically Bing, Yahoo, Startpage, Aol, and Ask—concurrently. By performing these searches asynchronously, it drastically reduces the time required to gather broad data points. The skill allows for highly granular control over the search process, including configurable page depth, proxy support, and automated output formatting (JSON, CSV, or text). Furthermore, it includes built-in post-processing capabilities such as result deduplication based on unique URLs or domain names, ensuring that the information returned is concise and relevant.

Installation

Installation is streamlined via the OpenClaw hub: clawhub install openclaw/skills/skills/orosha-ai/web-multi-search. Once installed, ensure you navigate to the skill directory and run the requirements.txt installer. It is critical to note that the core scraper must be installed directly from the GitHub repository using pip install git+https://github.com/soxoj/async-search-scraper.git --no-deps. This unique installation step ensures dependency compatibility, as the original library package naming may conflict with existing system libraries. For Linux users, a system-level dependency fallback is provided to ensure all async networking requirements are met.

Use Cases

This skill is indispensable for research-heavy workflows. Use it for market intelligence gathering where you need to verify if information is consistent across different search algorithms. It is perfect for developers needing to find technical documentation or obscure library issues that might be indexed differently on various search engines. It serves as a vital tool for OSINT (Open Source Intelligence) tasks where aggregating multiple data sources is required to build a complete profile or timeline of events. Additionally, it can be used for bulk data scraping to build datasets for training small-scale models or analyzing current trends in real-time.

Example Prompts

1. "Find the latest documentation and community discussions regarding the new updates to the async-search-scraper library using 5 pages per engine."
2. "Search for market competitor analysis for the term 'decentralized cloud storage' and save the results as a CSV file."
3. "Perform a broad search for 'best privacy practices in 2024' across all available engines and deduplicate by domain to get unique sources."

Tips & Limitations

To maintain high efficiency, always use the --unique-urls or --unique-domains flags if you are performing deep searches (e.g., more than 5 pages), as overlap between engines is common. While the tool is highly effective, remember that search engines may occasionally rate-limit your IP address; if you encounter connectivity issues, utilize the --proxy option to rotate your outgoing traffic. Always prioritize the JSON output format when building automated pipelines within OpenClaw, as it is the most robust format for parsing by the agent. Lastly, be mindful that excessive page requests can lead to CAPTCHAs; adjust your --pages setting moderately to maintain reliable operation.