xiaohongshu-scraper

What This Skill Does

The xiaohongshu-scraper is a specialized automation tool designed for OpenClaw agents to interface with Xiaohongshu (Little Red Book). It allows the agent to ingest content from platform URLs, providing a comprehensive data extraction service. Beyond simple scraping, it excels in downloading associated media assets (images), performing OCR (Optical Character Recognition) on visual content to extract text, and structuring the gathered information into clean, readable Markdown files or machine-readable JSON objects. It manages the lifecycle of local API services, ensuring that data retrieval is robust and persistent.

Installation

To integrate this skill into your environment, use the OpenClaw management command: clawhub install openclaw/skills/skills/ty-teo/xiaohongshu-scraper

Ensure that you have the necessary dependencies for the scraper and the underlying XHS-Downloader library. You must also initialize the local API service by running: ./scripts/xhs-api-service.sh start This service listens on port 5556 and acts as the bridge between your agent and the Xiaohongshu servers.

Use Cases

• Market Research: Aggregate top-performing posts for specific keywords or hashtags to identify current consumer trends.
• Content Curation: Automatically save high-quality guides, recipes, or travel itineraries from Xiaohongshu into your personal knowledge base in Markdown format.
• Competitor Monitoring: Track engagement metrics (likes, shares, comments) for specific authors or topics over time.
• Asset Management: Batch download original high-resolution images from posts for design or creative inspiration archives.

Example Prompts

1. "Please search for 'best minimalist desk setups' on Xiaohongshu, extract the content of the top 3 posts, and save them as a single Markdown file in my research folder."
2. "Go to this URL: [paste link here] and extract the text from the images using OCR, then summarize the core advice provided in the notes."
3. "Can you check the current engagement count for these five Xiaohongshu notes and tell me which one has the highest share-to-like ratio?"

Tips & Limitations

• Performance: OCR processing is resource-intensive. If you only need the text content from the post description, use the --no-ocr flag to significantly speed up retrieval times.
• Service Status: If the agent reports connectivity issues, verify that the API service is running by executing ./xhs-api-service.sh status.
• API Limits: Frequent mass scraping may trigger rate limits from the platform. Always respect the platform's terms of service and avoid aggressive, high-frequency requests.
• Data Privacy: Be mindful of privacy settings for specific notes; the scraper can only access publicly available information.