A Share Site Crawl

What This Skill Does

The A-Share Site Crawl skill is a specialized agentic tool designed for high-precision collection and normalization of financial data from China's primary A-share information ecosystems. It bridges the gap between raw web scraping and structured intelligence by providing dedicated workflows for 韭研公社, 雪球, 东方财富, 财联社, and 巨潮资讯. Rather than using generic scrapers, this skill employs context-aware access strategies, choosing between light-weight web_fetch for static content and browser rendering for dynamic, anti-bot-protected, or disclosure-heavy pages. It ensures that collected data is not merely a 'dump' of HTML, but a normalized record suitable for market analysis, sentiment tracking, or automated report generation.

Installation

To integrate this skill into your OpenClaw environment, use the following command: clawhub install openclaw/skills/skills/afengzi/a-share-site-crawl Ensure your agent has the necessary permissions for outbound network requests and local document processing if you intend to store the extracted data for long-term tracking.

Use Cases

• Automated Market Monitoring: Building cron jobs to pull daily telegraphs from 财联社 for real-time market event updates.
• Sentiment Aggregation: Tracking community sentiment and stock-specific discussions on 雪球 to identify retail investor trends.
• Regulatory Verification: Using 巨潮资讯 as a source-of-truth to verify corporate disclosure data extracted from less formal news aggregators.
• Research Normalization: Collecting disparate news items from 东方财富 and 韭研公社 into a unified, ticker-indexed format for investment reports.

Example Prompts

1. "Crawl the latest trending topics from 韭研公社 and create a summary of the current market narrative for the semiconductor sector."
2. "Perform a browser-first crawl on the latest disclosure documents for 000001.SZ from 巨潮资讯 and extract the core financial risk indicators."
3. "Set up a recurring workflow to collect the top 10 market news events from 财联社 every morning at 9:00 AM and normalize them into a JSON format."

Tips & Limitations

• Strategy Selection: Always default to web_fetch for static news pages to save resources, but pivot to browser when encountering sites like 东方财富 that utilize complex JavaScript to render market data tables.
• Error Handling: If a site returns a login wall or a dynamic captcha, do not attempt to bypass it; the skill is designed to report these as restricted to ensure data integrity.
• Data Integrity: Always refer to references/fields.md when defining your target schema to ensure ticker symbols and timestamps are correctly normalized across different platforms.